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OVERVIEW 


In September 2010, the Office of Head Start (OHS), in the U.S. Department of Health and 
Human Services’ Administration for Children and Families (ACF), awarded 17-month Early 
Learning Mentor Coach (ELMC) grants to 131 Head Start grantees. In March 2011, ACF’s 
Office of Planning, Research and Evaluation awarded a contract to American Institutes for 
Research, and its partners MEF Associates and the National Opinion Research Center at the 
University of Chicago, to conduct a descriptive study of the ELMC initiative. This study was 


guided by three key goals: 


Goal #1. Describe the implementation of the ELMC grants in HS programs. 

Goal #2. Examine the implementation factors of the ELMC efforts. 

Goal #3. Examine the factors that appear to be related to perceptions of successful 
coaching. 


This report provides detailed findings from: 


grantee census survey to collect information on a final respondent pool of 121 grantees 
(93 percent response rate); 

coach census survey to collect information on a final respondent pool of 384 coaches (84 
percent response rate); 

coach telephone interview with 54 coaches (83 percent response rate); and 

staff telephone interview with 80 staff members who received coaching (73 percent 
response rate). 


The study findings are presented according to seven practical aspects of coaching that are aligned 
to a conceptual framework of coaching in early care and education settings: 


context of coaching (e.g., size of grantee, population served, professional development 
resources); 

basic dimensions (e.g., goals of coaching, whom to coach, whom to hire as coaches, and 
how long to provide coaching) ); 

structural dimensions (e.g., logistics relating to where coaching will take place, coach and 
staff travel demands, scheduling, workload, and supervision of coaches); 

procedural dimensions (e.g., identifying staff needs, establishing staff goals, engaging in 
focused observation, providing feedback ); 

outputs of coaching (e.g., staff openness, coach-staff relationship); 

perceived outcomes of coaching; implementation successes and challenges; and 
sustainability of coaching program after the end of ELMC funding. 


The report concludes with a conceptual framework and implications for future research. 
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Executive Summary 


This study presented in this report describes the objectives, activities, approaches, strategies, and 
other aspects of the the Office of Head Start (OHS)’s Early Learning Mentor Coach (ELMC) 
initiative from the perspectives of HS grantees, coaches, and staff. In 2010, the Office of Head 
Start (OHS), within the Administration for Children and Families (ACF), announced the 
availability of funds to initiate coaching programs in a select number of HS grantees.’ Grantees 
were to use the funds to hire coaches who would then provide on-the-job guidance, training, and 
technical assistance to HS staff. This study was guided by three key goals: 


e Goal #1. Describe the implementation of the ELMC grants in HS programs. 

e Goal #2. Examine the implementation factors of the ELMC efforts. 

e Goal #3. Examine the factors that appear to be related to perceptions of successful 
coaching. 


The original goal of the ELMC initiative was to improve practices in HS programs. OHS received 
more than 280 grantee applications for the ELMC initiative, from which 130 grantees were chosen 
for funding for a 17-month project period. In October 2010, the funds to support the ELMC 
initiative were distributed in 42 states and the District of Columbia. The grant recipients reflected 
the diversity of HS programs, including Migrant and Seasonal Head Start and American Indian and 
Alaskan Native Head Start grantees. The grant awards ranged from $87,409 to the ceiling amount 
of $225,000; the average grant award was $215,513." 


Study Purpose and Design 


In March 2011, the Office of Planning, Research and Evaluation (OPRE) of ACF awarded the 
research contract to describe the coaching that occurred at HS grantees as a result of the ELMC 
initiative. The study team was also charged with developing a conceptual model that would help 
guide the study and that could be modified after the study to serve as a guide for both HS 
programs and researchers. We describe the resulting logic model in this report.” 


This descriptive study did not directly observe the coaching programs within ELMC grantees or 
draw conclusions about the impacts or effects of coaching on classroom, program, staff, or child 
and family outcomes. However, the findings of this study do provide rich descriptive 


" Although the ELMC initiative used the term mentor coach, for simplicity in this report, we use the term coach, 
except when referring to the data collection protocols, where the full name is used. 

* Originally, 131 grantees were awarded funds, but 1 grantee could not be included in the sample because it was 
unable to complete its project. By the end of the ELMC initiative, after data collection for the study was concluded, 
three additional grantees were unable to complete their ELMC projects. The list of the original 131 HS/EHS 
grantees that were originally awarded an ELMC grant in 2010 is in Volume 2: Appendixes, Appendix A. 

* A more in-depth description of the model and its components is in Putting the Pieces Together: A Program Logic 
Model for Coaching in Head Start. From the Descriptive Study of the Head Start Early Learning Mentor Coach 
Initiative (McGroder, Howard, Fishman, Rankin, & Helsel, 2013). 
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information, from both survey and interview sources, about coaching in HS programs. 


Achieving the three primary goals of this study required the quick design of an effective, 
comprehensive strategy to collect information across every level involved in the initiative: 
administrators, coaches, and staff members. To do this, the study employed a mixed-methods 
design using both quantitative (survey) and qualitative (interview) data collection and descriptive 
analysis methods. Data were collected from grantees, coaches, and staff during the last four 
months of the ELMC initiative. There were four data collection instruments in this study: 


e grantee census survey to collect information on a final respondent pool of 121 grantees 
(93 percent response rate); 

e coach census survey to collect information on a final respondent pool of 384 coaches (84 
percent response rate); 

e coach telephone interview with 54 coaches (83 percent response rate); and 

e staff telephone interview with 80 staff members who received coaching (73 percent 
response rate). 


Throughout the final report, univariate summary statistics from the grantee and coach surveys are 
presented in text, tables, bar charts, and pie charts. Thematic analyses from the coach and staff 
telephone interviews are presented in tables of major themes and illustrative quotes. 


Selected Findings 
The findings reported here are a brief selection from the overall report. 


Organization of report. The study findings are presented according to seven aspects of 
coaching that are aligned to the conceptual framework: 


e context of coaching (e.g., size of grantee, population served, professional development 
resources); 

e basic dimensions (e.g., goals of coaching, whom to coach, whom to hire as coaches, and 
how long to provide coaching) ); 

e structural dimensions (e.g., logistics relating to where coaching will take place, coach and 
staff travel demands, scheduling, workload, and supervision of coaches); 

e procedural dimensions (e.g., identifying staff needs, establishing staff goals, engaging in 
focused observation, providing feedback ); 

e outputs of coaching (e.g., staff openness, coach-staff relationship); 

e perceived outcomes of coaching; implementation successes and challenges; and 

e sustainability of coaching program after the end of ELMC funding. 


Context. Overall, the ELMC descriptive study found that a diverse group of HS and EHS 
grantees participated in the initiative, encompassing both large and small programs (ranging 
from serving fewer than 400 children to serving more than 5,000) in urban and rural settings 
(ranging from sparsely populated rural areas to urban areas with over 1 million residents). 


Basic Dimensions. 
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Goals. Grantees reported on their overarching goals for the ELMC grants and the qualifications 
for effective coaches. Goals commonly reported for the ELMC included improving classroom 
quality and staff practices, and addressing practices important for the Classroom Assessment 
Scoring System (CLASS). 


Timing of hiring. The timing of hiring, ratio of part time/full time, work hours, qualifications 
and other hiring decisions varied across ELMC grantees. On average, grantees hired their first 
mentor coach 2.4 months after the start of the grant; 72 percent of grantees began hiring within 4 
months from the grant start date. Perhaps one reason that many of the grantees were able to hire 
their coaches fairly quickly after being awarded the grant is that many used existing staff as 
coaches. Athough relatively few of the ELMC coaches hired (19.6 percent) were specifically 
coaches prior to the grant, many of the coaches hired were currently working for the grantee 
(57.1 percent) or had worked for the grantee in some other non-coaching capacity or had 
previously worked for the grantee in some non-coaching capacity (39.4 percent). 


Full-time/part-time. Grantees used ELMC funding to pay for a a variety of full-time and part- 
time coach positions. About one-third of the grantee respondents reported only part-time coaches 
and approximately half of the grantee respondents reported only full-time coaches. Most of the 
coaches worked part-time as coaches, although about one-third worked at least 40 hours a week. 
Approximately 20 percent of the coaches simultaneously held another job position with the 
grantee in addition to the coach role, and 43.9 percent of the coaches reported spending at least 
some time each week doing work for the grantee that was not part of their coaching role. 


Qualifications. The coaches as a whole were highly educated and had many years of ECE 
experience. Fifty percent of the coaches had 18 or more years of ECE experience. In contrast to 
their experience in ECE, most of the coaches did not have extensive experience specifically in 
coaching prior to ELMC initiative. Grantees ranked interpersonal skills as a key qualification for 
the success of coaching. Coaches themselves also rated interpersonal skills as the most important 
coach qualifications for success. In contrast, staff noted that a background in ECE work was the 
most important qualification for a coach (although interpersonal characteristics were also rated 
highly). 


Workload. About half of the coaches worked with 10 or fewer staff, whereas about one-fourth 
of the coaches worked with more than 20 staff. Coaches worked with a remarkably broad array 
of HS staff. Teachers and assistant teachers were the most common type of staff to receive 
coaching, but 19 percent of staff receiving coaching were home visitors and 18 percent were 
supervisors or administrators. Most of the coaches who worked with teaching staff; and most of 
the time they worked with both lead and assistant teachers. The coaches commonly worked in 
multiple centers; however, about one-fourth worked in only one center, while almost 40 percent 
worked in two to four centers and about one-third worked in five or more centers. 


Supervision role. About three-fourths of the coaches did not have supervisory responsibility for 
the staff they coached. However, almost all of the coaches reported to someone at the grantee 
level on the progress their staff was making. Almost all staff did not perceive coaches serving as 
supervisors or reporting to their supervisors as a problem. Some reported that it helped keep 
everybody “on the same page.” 


American Institutes for Research The Descriptive Study of the Early Learning Mentor Coach Grant Initiative—3 


Coaching Practice. 


Targeted topics. A key component of the coaching process is the focus of the individual 
coaching interactions. Coaches made decisions about topical coaching targets in varied ways. 
Most often, staff self-identified their coaching needs, and the coaches also identified staff needs 
by observing classrooms and staff, and using both formal and informal assessment methods. 
Coaches reported their top three targeted topics were (1) the general skills and strategies of the 
staff, (2) program and classroom operations, and (3) use of assessment and technology. Staff 
reported their own coaching goals as (1) improving the physical environment of the classroom, 
(2) improving teacher quality, (3) improving teaching of a school-readiness topic, (4) improving 
CLASS scores, and (5) providing behavior management techniques. 


Coaches roles. The coaches primarily perceived their role as being a collaborative partner with 
the staff they supported, while many also noted that they provide emotional support and 
instruction. The staff perceived the coaches as assistants, advocates, and sources of emotional 
support. The coaches were most likely to report using on-site observation, verbal feedback, and 
reflection strategies with each staff member at least three times in a typical month. Very few 
reported using video strategies. 


Perceptions of success. Grantees, coaches, and staff were asked about their perceptions of the 
coaching. Overall, the grantee administrators were very positive about the success of the 
coaching effort. Most of the coaches reported success in increasing staff openness to learning 
and improving the quality of practices. The staff reported changes in both instructional and 
behavioral management practices. Both the coaches and the staff reported that their relationships 
were supportive and open. 


Virtually all staff receiving coaching provided positive feedback on their experience. The staff 
noted that coaching was very effective, and reported overwhelmingly positive perceptions. Most 
of the staff were willing to continue coaching. Staff identified emotional support, availability, 
responsiveness, and constructive feedback as the most effective components of the coaching. The 
majority of the grantee administrators indicated that it was very likely they would continue to 
provide coaching after the ELMC grant ended. 


Challenges. There were implementation challenges reported by many grantee administrators. 
Both grantees and coaches reported that scheduling challenges as one of the biggest challenges to 
the success of the ELMC initiative: including demands on staff time and availability of substitutes. 
The grantee also included staff openness to improvement and staff level of engagement or interest 
as challenges. Of the potential challenges identified, the grantees were least likely to endorse 
challenges with coach turnover or the qualifications and abilities of the coaches. 


The majority of the coaches felt that the level of openness, the level of effort, the ability to 
engage in self-reflection and use feedback, and the ability of the staff to share mistakes were 
sometimes or often challenging to the coaches’ success.About one-third of coaches noted that at 
least sometimes, the level of support from the HS director could also be challenging. Coaches 
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were least likely to report challenges related to the relationship quality with staff. 
Limitations. 


This study has several limitations. It is not representative of all Head Start programs because the 
ELMC grants were awarded competitively to a small pool of Head Start programs. In addition, 
data collection started in the final months of the official grant period, so the study was limited in 
both the type of data it could collect and the research questions it could address. Given the design 
limitations, any findings in this report should not be interpreted as causal links, and caution should 
be taken when considering the applicability of the findings to Head Start more generally. 


Conclusion 


Program implications. The study provided a large amount of information about how coaching 
was structured and implemented in the OHS ELMC initiative. HS coaching programs can be 
initiated quickly, with highly educated coaching staff, and generally receive a positive reception 
by staff and administrators. Coaching can address both grantee-level goals and individual staff 
needs, and can contribute to an early childhood program’s quality improvement efforts. While 
coaching processes and approaches vary and also can be individualized for staff, this study found 
that the basic structure was remarkably similar across grantees. Coaching usually involved 
observation, modeling, and feedback strategies. Reports of the implementation challenges of the 
ELMC initiative were more varied across grantees. It would be important for a grantee to 
consider the logistical, administrative, and financial resources needed for any coaching effort. 
For more information about the important dimensions of coaching, see the more in-depth 
description of the HS coaching program logic model in Putting the Pieces Together: A Program 
Logic Model for Coaching in Head Start. From the Descriptive Study of the Head Start Early 
Learning Mentor Coach Initiative (McGroder, Howard, Fishman, Rankin, & Helsel, 2013). 


Research implications. The descriptive study of the ELMC initiative sought to describe the 
various aspects of coaching adopted by the ELMC grantees. The study findings raise additional 
questions and suggest fruitful areas for additional research. For example, it would be helpful to 
learn which coaching strategies tend to go together and whether these can be considered 
discernible models. Such information could inform future efforts to design and evaluate 
coaching in HS programs or other ECE settings. It would also be helpful to have a better sense of 
the range of expenses and costs to develop and implement a coaching program. This study did 
not gather any information about the costs of the program, the sources of money HS grantees 
may have been using in addition to their ELMC grant, resources for sustaining it, or any other 
cost information. 


It would also be useful in future studies of coaching to examine the coaching session more 
directly—to understand the experience of coaching; learn more about the process of coaching; 
and address questions about how coaching works, for whom it works, and under what 
circumstances it works. In addition, future research could use the program logic model from this 
study to inform empirical questions regarding the causal links between coaching and important 
outcomes. Within that logic model framework, examining the unique effects of different 
coaching dimensions on outcomes have not been not systematically examined to determine their 
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independent, additive effects or interactions. Little evidence exists on the effects of specific 
coaching dimensions on program, teacher, and child outcomes. 
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Introduction 


In 2010, The Office of Head Start (OHS) announced the availability of funds under the American 
Recovery and Reinvestment Act (ARRA) of 2009 (Public Law 111-5). Grantees were to use the 
funds to hire coaches who would provide on-the-job guidance, coaching, training, and technical 
assistance to Head Start (HS) and Early Head Start (EHS) staff.* The overarching goal of this 
Early Learning Mentor Coach (ELMC) initiative was to improve staff practices in HS programs. 
ELMC grant programs were established in October 2010. 


In March 2011, the Office of Planning, Research and Evaluation (OPRE) awarded a contract to 
evaluate the implementation of the ELMC initiative. The study team worked with OHS and 
OPRE staff and solicited advice from a group of experts who had expertise in early childhood 
mentoring and coaching, along with research and/or practice experience. The ELMC experts 
provided guidance throughout the study, including the development and revision of the 
evaluation framework, the study questions, the study design, the data collection plan and 
protocols, strategies for the analyses, the evaluation and program conceptual model, and final 
reporting. 


Study Purpose 


The purpose of this study was to describe the coaching that occurred at HS grantees as a result of 
the ELMC initiative. This provided a unique opportunity to describe the different dimensions of 
coaching within HS settings from the perspective of multiple stakeholders— administrators, 
coaches, and staff. Based on information gathered in the study, this report describes the various 
approaches to coaching implemented by ELMC grantees across context and basic dimensions (e.g., 
size of grantee, professional development resources, coaching goals, hiring coaches, qualifications 
of the coaches, and selecting staff to receive coaching); structural dimensions (e.g., logistics 
relating to where coaching will take place, coach and staff travel demands, workload); procedural 
dimensions (e.g., identifying staff needs and topical areas of coaching, coaching strategies, format, 
and frequency); and perceptions about coaching (e.g., the role of relationships, successes, 
challenges, and sustainability). The study team was also charged with developing a conceptual 
model that would not only help guide the ELMC study, but also could be refined based on the 
study to serve as a resource for both HS programs considering coaching and researchers tasked 
with evaluating coaching initiatives. 


The sample of programs included in this study is not representative of all HS programs or all 
types of coaching initiatives, as the ELMC grants were awarded competitively to only a small set 
of HS grantees. This study also did not directly observe the implementation processes of 
coaching programs within ELMC grantees or examine changes in HS staff over time. Therefore, 
the study cannot draw conclusions about the impacts or effects of coaching on classroom, 
program, staff, or child and family outcomes. Given these limitation, caution should be taken 
when considering the applicability of the findings to the broader HS program. However, the 


“In practice, Head Start and Early Head Start are referred to solely as Head Start. Therefore, this report follows that 
convention and uses the term Head Start inclusive of Early Head Start, unless distinction is warranted. 
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findings from this study do provide descriptive details about various aspects of coaching in early 
childhood settings. 


Organization of the Report 


This report contains eight sections. Coaching in Head Start and Early Childhood (page 9) 
provides a concise overview of the history of ECE professional development in general and 
approaches to coaching specifically. Study Design (page 16) outlines the mixed-methods design 
that was used to collect and analyze data; participant information; and the sampling, recruitment, 
and data collection procedures. The section Data Analysis Procedures (page 24) provides 
information about the overarching goals of the study and delineates the primary study questions 
within the three major goals of the study. Study Findings (page 27) reports the results of the data 
analyses across seven aspects of coaching that served as the conceptual framework for the study. 
These coaching aspects include grantee context, basic dimensions, structural dimensions, 
procedural dimentions, outputs, perceived outcomes, and sustainability. Near the end of the 
report, the section Summary of the Key Findings (page 74) provides a synopsis of the key points 
that emerged from the study data. In the section A Program Logic Model for Coaching (page 76), 
we present the program logic model that emerged from the study, which may be useful to HS 
programs considering a coaching initiative and researchers tasked with evaluating these 
initiatives. The final section, Suggestions for Future Research (page 79), outlines areas for 
additional research to inform the field of coaching in early childhood. 
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Coaching in Head Start and Early Childhood 


Several research studies have shown that ECE programs serving low-income children have been 
effective in narrowing early achievement gaps (e.g., Camilli, Vargas, Ryan, & Barnett, 2010; 
Gormley, Phillips, & Gayer, 2008; Heckman, Moon, Pinto, Savelyev, & Yavitz, 2010), but to 
ensure program quality, effective professional development for ECE staff—including training 
and coaching—is critical (Bogard & Takanishi, 2005; Zaslow & Martinez-Beck, 2006). 
Traditionally, coaching as a professional development strategy in ECE settings serves numerous 
purposes, including building the capacity of teaching staff for self-reflection, refining existing 
skills, implementing new skills, improving classroom environments, encouraging staff 
participation in continuing education, and promoting positive relationships among ECE staff 
(Gallacher, 1997; Rush & Shelden, 2005). 


Coaching and mentoring in ECE settings can take on many forms, including new teacher 
induction programs; informal relationships between junior and more experienced teaching staff; 
and formal mentor-protégé relationships that are more structured in terms of interaction 
frequency, duration, and nature. The emphasis of coaching may also vary between infant-toddler 
and preschool settings. For example, coaching for infant-toddler teachers is likely to have a 
strong focus on relationships (Edwards & Raikes, 2002), whereas coaching for preschool 
teachers is more likely to be focused on the implementation of curricula and/or teacher-child 
(Pianta, Mashburn, Downer, Hamre, & Justice, 2008). In addition, coaching and mentoring in the 
ECE field focus on a variety of topics important to young children, such as early childhood 
mental health, children’s socioemotional development, and school readiness skills. 


It is important to clarify the use of the terms mentoring, coaching, and consultation. There are 
distinctions among these terms, yet these three terms are often used interchangeably in ECE. The 
National Association for the Education of Young Children (NAEYC, 2011) provided definitions. 
Mentoring is described by NAEYC as a relationship-based practice between colleagues in 
similar professional roles, with a more experienced individual (the mentor) with knowledge and 
skills providing guidance to a less experienced individual (protégé or mentee).’ Coaching is also 
typically described as a relationship-based practice. But, in contrast to mentoring, it is led by an 
expert who serves in a different professional role than the recipient (coachee).° Consultation is 
described as a collaborative problem-solving process between an individual or group from one 
program and another organization. Consultation also facilitates the assessment of an issue to 
resolve it and/or to address a specific topic. 


This report uses the term coaching, but acknowledges that, in practice, mentor coaching, 


° See, for example, Sheridan et al. (2009). In addition, Cummins (2004) emphasized the relationship-based aspect of 
the mentor-protégé relationship, including the importance of rapport and trust between the learner and the mentor. 
Other researchers have suggested that as part of the relationship-based aspect, the mentor and the protégé can share 
knowledge and learn from each other (Hargreaves & Fullan, 2000; Whitebook & Bellm, 1996). 

° Rush and Shelden (2005) define coaching as “an adult learning strategy where a coach promotes a learner’s ability 
to reflect on his or her actions as a means to determine the effectiveness of an action or practice and develop a plan 
for refinement and use of the action in immediate and future situations” (p. 3). 
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mentoring, consultation, facilitation, and even the broader on-site professional development and 
technical assistance can overlap with coaching in the ECE field. Research suggests that there is a 
great deal of variation in what is referred to as a mentoring or coaching activities, but overall 
these terms refer to a broad range of professional development strategies (Edwards, 2003). There 
is no established typology for an ECE coach and the process for working with ECE 
professionals. However, a basic coaching model typically involves a cycle of initial assessment, 
goal setting and planning, modeling by the coach, scheduled coach observations of staff practice, 
and opportunities for reflection (e.g., Dunst & Trivette, 2009; Rush & Shelden, 2005; Snyder & 
Wolfe, 2008; Snyder et al., 2012). The National Center on Quality Teaching and Learning 
(NCQTL), one of the five technical assistance centers that OHS established in 2010, developed a 
practice-based model of coaching that involves several of these practices, including planning and 
goal setting, observations of practice, reflection, and feedback (Snyder et al., 2012). 


A small, but growing, research literature suggests overall positive effects of various types of 
coaching on teaching (e.g., Dunst & Trivette, 2009; Miller, 1994; Neuman & Cunningham, 
2009; Rudd et al., 2009; Villar & Strong, 2007) and child outcomes (e.g., Biancarosa, Bryk, & 
Dexter, 2010; Fukkink & Lont, 2007; Powell & Diamond, 2011; Powell, Diamond, Burchinal, & 
Koehler, 2010). However, less attention has been paid to the implementation of specific 
components and processes that are associated with the highest quality and differential success of 
coaching programs. There is a need to better understand the characteristics and dimensions of 
coaching programs and the contextual factors that might affect such professional development 
interventions within research and evaluation studies (Zaslow, 2009), for translating what we are 
learning from these studies to practice and policy-related decisions. 


History of Coaching in Head Start 


Mentoring and coaching in HS programs have a rich and varied history, with programs 
implemented at national, state, and local levels. Some efforts were initiated by OHS, whereas 
others are broader programs targeted to the ECE field. The philosophy of mentoring and 
coaching aligns with HS philosophy and its practices concerning professional development. As 
described in Putting the Pro in Protégé, an OHS guide to mentoring (U.S. Department of Health 
and Human Services, 2001), mentoring is consistent with numerous HS goals, including the 
requirement that grantees implement a formal approach to staff training and development. The 
value of mentoring and coaching was first recognized within the Head Start Act of 1998, and 
similar language was also contained in The Improving Head Start for School Readiness Act of 
2007 (Sec. 635. [42 U.S.C. 9801]. These HS Acts specified resources for supporting mentors, 
and defined a mentor teacher as follows: 


An individual responsible for observing and assessing the classroom activities of a Head 
Start program and providing on-the-job guidance and training to the Head Start program 
staff and volunteers, in order to improve the qualifications and training of classroom 
staff, to maintain high-quality education services, and to promote career development in 
Head Start programs. (Sec. 648A (b)(1)) 


The Federal government has funded a variety of initiatives, resources, and supports related to 


mentoring and coaching in ECE settings. Selected initiatives, including technical assistance 
resources and research studies, are listed in Exhibit 1. 
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Exhibit 1. Selected Federally Funded Mentor, Coaching, and Consultation Initiatives in Early 
Childhood Education Settings 


Date a i:) Funder Description Summary and Internet Links 

2001 | Putting the Pro in OHSa Technical This resource includes information on the principles of 
Protégé: A Guide assistance mentoring, its role in HS programs, strategies for 
to Mentoring in resource implementing effective programs, and a variety of examples of 
Head Start and mentoring efforts implemented by HS grantees across the 
Early Head Start United States. 

http://eclkc.ohs.acf.hhs.gov/hslc/hs/resources/ECLKC_Bookst 
ore/PDFs/pro_in_protege.pdf 

2002 | Strategic Teacher OHS Technical This teacher training program was designed to promote 
Education assistance evidence-based early literacy strategies that would be 
Program (STEP) resource and | delivered by trained mentor teachers to their peers. Following 
Early Literacy training centralized training, the early learning coaches were 
Mentor Coach? program supported at their grantee sites by coach specialists through 

quality improvement centers. 

2003 | Interagency OPRE, NICHD, | Research Three of the School Readiness Consortium grants focused on 
School Readiness and ASPE of educational improvement efforts that incorporated teacher 
Consortium HHS; OSERS of training and ongoing support, e.g., through mentoring or 

the U.S. coaching, of fidelity in teacher implementation of treatment. 
Department of Targeted content for the three projects included 
Education socioemotional development, language-emergent literacy, and 
teacher-child interaction. 
http://www.acf.hhs.gov/programs/opre/research/project/interaq 
ency-school-readiness-consortium-2003-2008 

2005 | Steps to Success: OHS Technical This program is designed to support early literacy coaches in 
An Instructional assistance their work with grantee staff. It provides a multimedia 
Design for Early resource professional development system to support early literacy 
Literacy Mentor coaches and other mentoring efforts in HS programs. Several 
Coaches in Head resources are part of Steps to Success, including a facilitator’s 
Start and Early guide, a coach manual, a protégé’s journal, a professional 
Head Start development plan, a decision-maker’s guide, and DVDs. 

http://eclkc.ohs.acf.hhs.gov/hslc/hs/resources/video/ST S/testi 
ngsts.htm 

2008 | Early Reading U.S. Department | Resources This program was designed to support early childhood centers 
First of Education | and research | by providing professional development resources for teachers, 

study educational materials for high-quality learning environments, 
teaching strategies, and assessments based on scientific 
research. 
http://www2.ed.gov/programs/earlyreading/index.html 
http://www.studentsupportcenter.org/EarlyReadingFirst/ 

2008 | Head Start OPRE Research Each of these seven grants investigated the effectiveness of 
University study coaching, mentoring, and consulting with HS teachers. Each 
Partnership study developed its own professional development program 
Grants: Strategies and gathered implementation and outcome data regarding 
for Improving the coaching processes and contextual and organizational factors. 
Effectiveness of http:/www.acf.hhs.gov/sites/default/files/opre/head_start_univ 
Head Start ersity partnership research grants strategies for.pdf 
Teachers 
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Title 


Date | 


Description Summary and Internet Links 


2010 | National Center on OHS Technical This center focuses on the best practices for HS teachers, 
Quality Teaching assistance with a special emphasis on the transition to kindergarten. It 
and Learning resource provides teacher development through a variety of means, 
(NCQTL) including technical assistance, study groups, and mentoring. 


http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/teaching/center 


aFormerly the Head Start Bureau. "This program no longer exists, and an active Web link is not available. A 2003 report authored by Melinda Gish through the 
Congressional Research Services states, “The Administration has moved ahead with two additional efforts that are in keeping with the Good Start, Grow Smart 
initiative, but that do not require legislative changes to the Head Start Act. One is the Strategic Teacher Education Program, also known as Project STEP, 
described by the Head Start Bureau as ‘a comprehensive, multi-faceted, sequential professional development endeavor to ensure teachers use research-based 
strategies to implement early and emergent literacy.’ As part of this development, during the summer and fall of 2002, 3,000 Head Start staff and 100 state child 
care administrators received 32 hours of training in strategies to support children’s emerging literacy. Those who were trained are expected to serve as ‘mentor 


coaches’ for staff within their respective Head Start programs” (p. 12). 


In addition to these Federally funded coaching initiatives providing technical assistance, 
resources, and/or research on the topic, numerous coaching initiatives exist at the local level in 
HS and other ECE programs. Grantee-level coaching initiatives include programs to support new 
teachers as they enter the program and to improve classroom quality. The ACF publication 
Putting the Pro in Protégé (2001) offers useful descriptions of several examples of local 
coaching programs implemented by HS grantees, including the goals of each initiative; the 
qualifications of the mentor staff; the mentor selection process; and other aspects of the mentor- 
protégé relationship, such as the frequency and duration of their partnership. Examples of other 
local and state early childhood mentoring initiatives are given in Exhibit 2. 


Exhibit 2. Examples of Local and State ECE Coaching Initiatives 


Date Title Funder Description Summary and Internet Links 
n.d. | Day Care San Francisco’s | Clinical services | Run out of San Francisco's General Hospital, this program 
Consultants Community and training provides consultation to support child care staff and 
Infant-Parent Behavioral Health | program program administrators to build their capacity to ensure the 
Programs Services healthy growth and development of children; it includes 
observation of case children and families, consultation with 
staff, and direct work with families. 
http://www. infantparentprogram.org/ 
1988 | California Early | Initial funding Technical Originally started at Chabot College and now operated out 
Childhood from the United —_| assistance of the City College of San Francisco, this program provides 
Mentor Program | Way ofthe Bay | resource and Stipends and training to mentor teachers, who then work 
Area and the training program | with ECE students at the community college level, and 
David and Lucile director mentors who provide support to other ECE 
Packard program directors. 
Foundation@ http:/Avww.ecementor.org/ 
1994 | Early Childhood | Multiple funders® | Consultation, This program, which is affiliated with the University of 
Partnerships mentoring, Pittsburgh Schools of Education and Medicine, includes 
technical developmental health care mentors who partner with child 
assistance, and | care staff and parents to address family and child needs. 
research http://earlychildhoodpartnerships.org/index.cfm 
collaborative 
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Title Description Summary and Internet Links 
1997 | Day Care Plus Cuyahoga Technical This program, also known as the Early Childhood 
Positive County's Invest in | assistance Consultation, is a part of Cleveland’s Positive Education 
Education Children through | resource and Program and offers intensive consultation (e.g., 20 hours a 
Program Starting Point for | training program | week for 2 years) in ECE settings, including observations, 
Child Care and referrals, interventions, and staff training. 
Early Education http://pepcleve.org/day.aspx 
Late | Michigan Child | Initial funding Mental health This program includes consultation for children with 
1990s* | Care Expulsion | from the consultation, challenging behaviors, including intervention with case 
Prevention Department of intervention, and | children; training for administrators, staff, and parents to 
Program Human Services; | training program | build their capacity to promote children and families’ 
Michigan mental health; and strategies to improve communication 
Department of among administrators, staff, and parents. 
Community http://www.michigan.gov/mdch/0,1607,7-132- 
Health 2941 4868 7145-14785--,00.html 
2000 | The Capital Area | Harrisburg Center | Research study | Run through an initiative of Penn State University, the 
Early Childhood | for Health Child | with focus on intensive, on-site mentoring to infant and toddler 
Training Institute | Development and | technical caregivers in the study attempts to fill the gaps they 
Mentoring the Capital Area | assistance and _| perceive are inherent in the more typical workshop-training 
Program Funders Group | mentoring model. 
program http://ecti.hbg.psu.edu/for_providers/ment_intro.htm 
2004 | Chicago Schoo! | National Institute | Research study | Developed through the Steinhardt School at New York 
Readiness of Child Health —_| with focus on University, this classroom-based intervention includes a 
Project and Human classroom- coaching component in HS grantees and intensive mental 
Development, the | based health consultation, in addition to training provided to 
William T. Grant | intervention teachers. 
Foundation, and http://steinhardt.nyu.edu/ihdsc/research/csrp/ 
the McCormick 
Tribune 
Foundation 


aSubsequent funding was provided in 1991 by the American E 
Care and Education at Wheelock College, the Clorox Compan 


xpress Philanthropic Program, the ARCO Foundation, the Center for Career Development in Early 
y Foundation, the Conrad N. Hilton Foundation, the David and Lucile Packard Foundation, the San 


Francisco Foundation, and the Morris Stulsaft Foundation. In 1992, the program contracted with the California Department of Education for funding from the 
Federal Child Care and Development Block Grant. *According to their website, “ECP has partnered successfully with over 300 local, regional, national, and 
international agencies since 1994.” ‘An exact date for the start of this program is unavailable. 
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The Head Start Early Learning Mentor Coach Initiative 


The ELMC grantees were awarded approximately $25 million to support coaches who would 
provide on-the-job guidance, coaching, training, and technical assistance to HS staff. OHS did 
not prescribe a specific coaching model for this initiative, so the grantees developed their own 
coaching approaches to best meet their unique needs while also being true to the overarching 
goals of the initiative. OHS received more than 280 grantee applications for the ELMC initiative, 
from which 130 grantees were chosen for funding for a 17-month project period.’ The grantees 
that were awarded ELMC funds reflected the diversity of HS programs, including rural and 
urban; large and small; and monolingual, bilingual, and multilingual programs. The HS grantees 
also included center-based programs, home-based programs, family child care, and child care 
partners (Head Start Resource Center, n.d.). The ELMC grantees also included American Indian 
and Alaska Native (AIAN) and Migrant and Seasonal Head Start (MSHS) programs. Funds to 
support the ELMC initiative were distributed in 42 states and the District of Columbia. The grant 
awards ranged from $87,409 to the ceiling amount of $225,000; the average grant award was 
$215,513.° The ELMC initiative operated from September 2010 to February 2012, with a 
number of grantees extending their project up to one year longer without additional funding. 


In the original funding opportunity announcement, OHS guidance to the ELMC applicants was 
broad, and suggested the following (OHS, 2010): 


e Coaches should provide support for teaching staff to improve the development and 
the early learning of HS children. 


e Coaches should focus on clearly stated outcomes that target improving instructional 
outcomes as measured by CLASS scores and the Head Start Child Outcomes 
Framework. 


e Coaches should provide ongoing and consistent assistance to teaching staff over a 
period of time, with the amount of time devoted to working with each staff member 


determined by assessing the strengths and needs of each individual. 


OHS suggested some example coaching activities that the grantees could include as part of their 
ELMC programs (Office of Head Start, 2010, p. 3): 


e “Establishing a comfortable relationship with assigned teaching staff” 
e “Leading discussions on early child development and learning” 


e “Assisting teaching staff with appropriate curriculum planning and implementation” 


’ Originally, 131 grantees were awarded funds, but 1 grantee could not be included in the sample because it was 
unable to complete its project. By the end of the ELMC initiative, after data collection for the study was concluded, 
three additional grantees were unable to complete their ELMC projects. 

* The list of the original 131 HS/EHS grantees that were originally awarded an ELMC grant in 2010 is in Volume 2: 
Appendixes, Appendix A. 
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e “Conducting observations of teaching staff and of their interactions with children and 
families to create opportunities for discussion” 


e “Providing feedback in a supportive, instructive and non-judgmental way” 


e “Enhancing the ability of teaching staff to observe children's activities and 
experiences as part of the ongoing assessment process and to use this information to 
individualize the curriculum” 


e “Modeling appropriate strategies for engaging children in interactions designed to 
improve their vocabularies, content knowledge, love of learning and persistence in 
pursuing inquiries” 


e “Providing opportunities for participants to reflect on and apply theory and learning 
to everyday practice” 


There was a large potential for variation within the ELMC initiative across the HS grantees that 
received the awards. Information provided by OHS and an initial review of the funded grantee 
proposals revealed that the ELMC grantees proposed to work with various types of HS staff, 
including lead teachers, assistant teachers, home visitors, family child care providers, and 
supervisors. Some of this variation was consistent with the grantees’ multiple program options: 
center-based care, home-based care, family child care, and a combination option, which provided 
services to children both in a center-based setting and through home visits. In addition, a range of 
organizations operated the HS grantees, including school districts, community-based agencies, 
universities, and child care centers. Finally, as noted earlier, OHS did not prescribe a specific 
coaching model or approach for the ELMC initiative. 


Early Learning Mentor Coach Study Purpose and Timing 


The study aimed to describe what approaches HS grantees used to provide coaching in their 
programs and to examine the features of the implementation of these efforts. The study team was 
also charged with developing a conceptual model to help frame the descriptive study of the 
ELMC initiative and determine research questions related to the three overarching project goals 
originally put forth by OPRE and OHS: 


e Describe the implementation of the ELMC grants in HS programs. 
e Examine the implementation factors of the ELMC efforts. 
e Examine the factors that appear to be related to perceptions of successful coaching. 


The information collected in this study is extensive and achieves the primary purposes of 


describing many practical details of the initiative from the viewpoints of the primary 
stakeholders: grantee administrators, coaches, and program staff. 
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Study Design 


Achieving the goals of this study, within the time constraints of the ELMC initiative, required the 
rapid design of an effective comprehensive strategy to collect information across the 
administrators, coaches, and program staff members who were involved in the initiative. Thus, the 
study utilized a mixed-methods design that employed both quantitative and qualitative data 
collection and analysis methods. 


Exhibit 3 lists the major research questions aligned to the study’s original three goals. For each 
research question, both quantitative (survey) and qualitative (interview) methods were used to 
collect data that could serve different, but complementary, purposes. Quantitative survey 
methods were used to examine the prevalence of an array of coaching features among the 
grantees and the coaches, and our sampling strategies allowed us to describe the ELMC 
population of grantees and coaches and to make generalizations that are beyond simple, chance 
occurrences. However, by design, quantitative surveys limit the ability to capture open-ended 
perceptions of process and context. Thus, qualitative methods were used in conjunction with 
quantitative methods to explore contextual details concerning the experiences and perceptions of 
coaches and staff. Obtaining in-depth perspectives about the ELMC initiative—implementation, 
helpful factors, staff changes, staff willingness, program expectations, program successes, 
program challenges, and perceived critical elements of the ELMC initiative—was possible 
through the use of qualitative approaches. These methods also allowed for triangulation of the 
data to answer all of the research questions listed in Exhibit 3 from multiple perspectives, 
providing a more nuanced and context-specific examination of the ELMC initiative. 


Exhibit 3. Study Goals and Research Questions 


Goal #1: Describe the Implementation of the ELMC Grants in HS Programs 


RQ1: What were the key features of the coaching program model or approach? How was the grantee structuring the coaching 
initiative? 


RQ2: What were grantees doing to integrate and sustain coaching beyond the grant period? 
RQ3: What was the perceived quality of the coaching as it was being implemented on the ground? 
RQ4: Were staff ready, willing, and able to participate in the coaching as expected? 

RQ5: How were staff members changing their behavior as a result of coaching? 

RQ6: What were the perceived successes and challenges in implementing the coaching? 


RQ7: What features—such as the characteristics of coaches, OHS resources, the coach-staff relationship, staff behaviors and 
attitudes, teacher characteristics, and organizational characteristics—play in the implementation and the perceived success of 
coaching? 


RQ8: What did the staff find the most and the least helpful about coaching? 
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To address these study goals and research questions, using available theoretical and empirical 
research and with input from the ELMC technical consultant group comprised of ECE experts on 
coaching and mentoring, we identified several aspects of coaching to describe in the study, as 


detailed in Exhibit 4. 


Exhibit 4. Aspects of Coaching Examined in the ELMC Study 


Topic Description 


Context of Coaching 


Grantee characteristics 


context 


Grantee and community 


Includes characteristics such as program type, program size, and the population they serve 
Context of the grantee and the grantee’s community, such as availability of professional 
development resources in the community, their history of providing professional development, and 
their use of technical assistance resources. 


Basic Dimensions 
Coaching goals 


The overarching goals and the priority areas of a grantee’s ELMC initiative, such as including 
improving classroom quality, the quality of staff practices with children, and improving CLASS 
assessment scores 


Staffing of coaches 


When coaches were hired, whether hired part-time or full-time, and past experience working with the 
grantee 


Coach background and 
qualifications 

Staff targeted for 
coaching 


How centers and staff were selected for coaching, types of staff receiving coaching (e.g., teachers, 


Coaches’ education level and degrees/certifications, experience in ECE and/or coaching adults, 
personal characteristics, and importance of coach qualifications 


assistants, home visitors) 


Structural Dimensions 


Training of coaches 


Supervision of coaches | Whether coaches were supervised by grantee staff, which grantee staff served as supervisors, how 


Whether coaches received an orientation to the ELMC initiative and/or ongoing training, the content 
of this training, and coach satisfaction with training 


often coaches met with their supervisors, and topics discussed during these meetings 


Coach workload 


Information about the employment status, hours worked, the number of programs served, the type 
and number of staff served, and the coach-to-staff ratio 


Procedural Dimensions 


Identifying staff needs 


How coaches determined coaching needs of staff, such as use of assessment tools, observing 
staff, and having staff self-identify needs 


Establishing staff goals 


Typical staff goals, such as increasing staff knowledge, skills, and strategies; improving structure 
and organization of the classroom; increasing the use of assessment or technology; and 
encouraging professional growth (such as obtaining a degree) 


Topical areas in 
coaching sessions 


The topical areas covered in the coaching sessions, including increasing staff skills and strategies, 
increasing staff professional knowledge, and encouraging staff personal growth 


Interacting with staff 


The practical format (such as phone or face-to-face meeting) and frequency of interacting with staff 


Roles of Coaches 


Roles that coaches frequently served with the staff they work with, including collaborative partner, 
emotional supporter, teacher/instructor, and staff supervisor 


Coaching strategies 


Strategies used to observe staff behavior, to share different approaches for working with children 
and families (such as modeling), and to provide feedback and support to staff 


Outputs of Coaching 


Staff openness to and 
engagement in coaching 


Coach-staff relationship | Staff opinions on the degree to which their coach was skilled and knowledgeable in areas helpful to 


Coaches’ ratings and illustrative examples of staff openness to and engagement in coaching, 
including staff's positive attitude, willingness to consider coaches’ suggestions, and efforts to 
implement suggested changes 


them, and some examples 
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Topic Description 

Perceived Outcomes of Coaching, Successes and Challenges 
Perceived success and | Grantee views on coaches’ success in a variety of areas and in meeting overall goals of ELMC 
changes as a result of initiative, coaches’ views of their own success overall and in fostering changes with particular staff 
coaching 

Perceived Challenges Grantee and coach perceptions of challenges implementing ELMC initiative; staff perceptions of 
challenges participating in coaching 
Factors associated with | Factors perceived to foster success or are associated with challenges in coaching, such as coach 


successes and characteristics, staff characteristics and coaching strategies 

challenges 

Sustainability 

Sustainability of Grantee plans for continuing their coaching program after the ELMC grant funding ended 


Coaching programs 


The study findings are presented using these key aspects of coaching as a framework (findings 
start on page 27). 


Sampling and Recruitment Procedures 


To collect information that documented the range of coaching approaches among the ELMC 
grantees in the final months of the ELMC initiative, the data collection activities included online 
census surveys and telephone interviews with three types of respondents (Exhibit 5): grantee 
administrators (e.g., HS program administrators, directors, or ELMC grant directors), ELMC 
coaches, and staff receiving coaching (e.g., classroom-based staff, home-based visitors, family 
child care providers, or other types of staff). 


Exhibit 5. Study Participants and Data Collection Instruments 


Type of Study Targeted Number Data Collection Instrument 
Participant of Participants 
Grantees (all) 130 Grantee census online survey 
Coaches (all) 455 Coach census online survey 
Sample of coaches 65 Coach telephone interview 
Sample of staff 130 Staff telephone interview 


The entire ELMC grantee and coach populations were recruited to complete the surveys. 
However, for the qualitative telephone interviews, a target sample of 65 grantees was chosen, 
from which 65 coaches (one from each sampled grantee) and 130 staff (two from each coach) 
were selected. This target number ensured that, by representing 50 percent of the grantees, we 
would capture sufficient variation in the grantees. 


A priority was that the sample selected for the qualitative telephone interviews be balanced 
across a range of grantee characteristics. This required considerable attention to the grantees, 
because each grantee may have a slightly different structure that may influence the perspectives 
provided in the interviews about the initiative. For purposes of this study, we selected carefully 
across the types of programs (EHS, HS, or combination), size (the number of centers and the 
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number of children), geographic location (urban, rural, or suburban), and populations served.” 


To select a sample of 65 interview grantees from the population of 130 grantees, we relied on a 
three by four (rural or urban classification by grantee size) stratified sample design. Exhibit 6 
shows the population and the sample sizes within each stratum of this sampling design. First, in 
order to ensure inclusion of types of programs of interest, we selected all of the large grantees 
(i.e., grantees with greater than 5,000 children) and all of the AIAN and MSHS grantees, which 
resulted in a total of 15 grantees. These are represented in the shaded boxes. The design allocated 
the remaining sample of 50 grantees across the unshaded strata in proportion to the number of 
grantees within each stratum that were not selected with certainty (referred to as noncertainty 
grantees). Second, to select the sample within each unshaded stratum, we relied on a systematic 
sampling procedure. The procedure sorted the records of all noncertainty grantees within each 
unshaded stratum by program type (EHS, HS, and HS/EHS), the proportion of children who are 
DLLs, and program options (whether the program is entirely center-based or offers other options, 
such as family or home-based child care) before the sample of noncertainty grantees was drawn. 
The procedure ensured that the sample of noncertainty grantees drawn for the remaining sample 
of 50 was balanced across the identified background characteristics (i.e., type, size, location, 
DLL population). '° 


Exhibit 6. Sample Design for the Early Learning Mentor Coach Study 


Population Size (Number of Noncertainty Grantees) 


Sample Size [Number Selected Randomly, Number Selected With Certainty] 


Size 
Less Than 400 to 1,000 | 1,001 to 5,000 More Than Total 
400 Funded Funded Funded 5,000 Funded 
Children Children Children Children 
Metro area with one 17 (15) 15 (14) 22 (22) 3 (0) 97 (51) 
million or more residents 
(large urban) 8 [6, 2] 7 [6, 1] 10 [10, 0] 3 [0, 3] 28 [22, 6] 
Seer Metro area with less 7 (7) 11 (11) 20 (18) 1 (0) 39 (36) 
classification than one million 
residents (small urban) 3[3, 0] 5 [5, 0] 10 [8, 2] 1 [0, 1] 19 [16, 3] 
20 (17) 12 (12) 2 (0) 1 (0) 35 (29) 
Rural area 
10[7, 3] 5 [5, 0] 2 [0, 2] 10, 1] 18 [12, 6] 
44 (39) 38 (37) 44 (40) 5 (0) 131 (116) 
Total 
21 [16, 5] 17 [16, 1] 22 [18, 4] 5 [0, 5] 65 [50, 15] 


° The source of grantee’s descriptive information for sampling was the Head Start Program Information Report 
(PIR) administrative data set. 
'° Additional details about the sampling procedures, including the base weights of the program type, the proportion 


of children who were DLLs, and program options can be found in Volume 2: Appendixes, Appendix C. 
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To initiate the recruitment process, ELMC grantees were contacted and asked for contact 
information for their current list of coaches. Based on the coach lists provided by each grantee, 
coaches were selected for the telephone interviews. Based on staff lists provided by the selected 
coaches, two staff members per selected coach were also recruited to participate in the telephone 
interviews. 


Data Collection Instruments 


Four instruments collected descriptive information about initiative experiences from the 
population of ELMC grantees, the population of coaches, and from a matched sample of coaches 
and the staff members they coached. These protocols were developed based on a literature 
review on coaching, an evaluation conceptual model that was developed to help frame the study, 
and input from the project’s expert consultants, OPRE staff, OHS staff, and HS program 
administrators. '* Draft versions of the protocols were pilot tested with nine HS grantees to gather 
their feedback related to length, the terminology used, the response items, understandability, and 
value. The final data collection protocols were as follows: 


Grantee census survey. This 40-question online survey required approximately 30 
minutes for the responders to complete. It was designed to collect descriptive data about 
the overall approach to professional development used by the grantees (i.e., the 
professional development context), the goals and objectives, the operation of the ELMC 
initiative, the coaching approach and implementation, any perceptions about the 
effectiveness of coaching, any reflections about the challenges of coaching, and the plans 
for sustaining coaching. 

Mentor coach census survey. This 63-question online survey required approximately 
30 minutes for the responders to complete. It was designed to collect descriptive data 
about the background and the experience of the coaches, preparation for the ELMC 
initiative, the approach to coaching, the goals and the content for coaching, any 
perceptions about the effectiveness of coaching, and challenges and facilitating factors 
about coaching. 

Mentor coach telephone interview. This 60-minute interview gathered in-depth 
information from a selected sample of coaches about their experiences in ECE and 
coaching; their goals and the content used for coaching; their coaching approach in 
general and with two specific staff members whom they coached; and additional 
reflections about the coaching experience, including elements of perceived success and 
challenges. The interviews provided additional contextual details about their experiences 
and work with the staff. 

Staff telephone interview. This 60-minute interview gathered in-depth information 
from a selected sample of staff members who were coached by the interviewed coaches, 


"' Additional information about the recruitment procedures can be found in Volume 2: Appendixes, Appendix C. 


12 


The working evaluation logic conceptual model was an internal working deliverable for the contract and had two 


purposes: to guide the study and develop a revised program conceptual model. The final conceptual model is 
presented at the end of this report in Exhibit 60 (page 82) and described in a separate document developed as a part 
of this project: Putting the Pieces Together: A Program Logic Model for Coaching in Head Start. From the 
Descriptive Study of the Head Start Early Learning Mentor Coach Initiative (McGroder et al., 2013). 
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including information about their work and professional development; their experiences 
with coaching implementation and approach; the goals and the content of the coaching 
received; their experiences in a typical coaching session; their perceptions of the coach; 
their perceptions about the effectiveness of coaching; and any reflections about the 
coaching experience, including any challenges. The staff interviews provided 
information directly from the perspectives of those coached, which was not available in 
the surveys. 


The audio of both staff and coach interviews was recorded. In addition, interviewers completed a 
data capture form at the end of the interview, which summarized the key interview responses. 
The data capture form was a way for the interviewer to take notes in a systematic manner, and it 
provided an efficient and organized way to review the interview data and compose a post- 
interview narrative reflecting the interview. 


Data Collection Procedures and Response Rates 


The evaluation team administered the grantee census surveys and coach census surveys 
electronically using a secured, Web-based software program called Vovici. Unique links for each 
invited grantee and coach were distributed by e-mail. This Web-based technology enabled the 
respondents to complete the survey when convenient, and in one or multiple sittings. 


Telephone interviews were conducted with the coaches and the staff members they coached. The 
interviewers were trained to ensure the consistent collection of accurate and reliable data from 
the respondents. The trainers and the data collectors were experienced and well versed in 
interview techniques, were familiar with the ELMC study, and had knowledge of the ECE 
population from whom the data were collected. Exhibit 7 details the final response rates for both 
types of data collection procedures. 


Exhibit 7. Response Rates 


INTUTan) os) ace) exe dossy 0) (2 


Participants Number of Participants Percentage 


Surveys 


Grantee census survey 
Coach census survey 
Interviews Number of Participants | Percentage 


Seinen 


Coach and staff interview matched pairs 


54 49 91% 
(at least one staff per coach) 


Item-level response rates for the survey questions were generally high for most of the survey 
questions, although the range of the response rates was wide. For the grantee survey, the item 
response rates ranged from 40.5 percent to 100 percent, with an average response rate across the 
questions of 97.3 percent. For the coach survey, the item response rates ranged from 54.3 percent 


'S Additional information about the quality assurance procedures for the online surveys is in Volume 2: Appendixes, 
Appendix E. 
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to 100 percent, with an average response rate across the questions of 95.5 percent. Note that 
many of the survey questions that were skipped by respondents asked about staff types or 
program resources that were not utilized by all grantees (e.g., regarding participation of family 
child care staff). Instead of marking these items as non-applicable or zero, it is likely that 
respondents simply skipped them.“ Validity checks were conducted for multipart questions, and 
we found no evidence that the survey respondents selected the same response (such as yes or 
very helpful) for every subquestion. 


Limitations of the Study 


This study provided descriptive information about coaching in 130 HS grantees that were 
awarded ELMC grants. The surveys for this study were conducted with the full population of the 
ELMC grant recipients and their coaches. However, the study is not representative of all HS 
programs because the ELMC grants were funded from one-time ARRA funds and were awarded 
competitively to a small pool of HS grantees that applied for the grant. Thus, the ELMC grant 
recipients are likely to be different from the larger population of all HS programs, and findings 
from this study are not generalizable to all. 


Although the study has a strong overall response rate, particularly for the grantee and coach 
census surveys, an additional limitation is that some of the individual survey items had low 
response rates. Further, the staff interview response rate was lower. One reason for the lower 
response rate among staff is because the data collection period coincided with the end of the 
ELMC funding period for many of the grantees.'° As such, grantees administrators may have 
been less responsive in helping to follow-up with staff selected to be interviewed, or program 
staff themselves believed participating in the interview was no longer relevant if their ELMC 
project had ended. It may be that staff who were more positive about their coaching experience 
where quicker to respond to our recruiting efforts. Regarding of the potential reason, it is 
important to note that non-response bias may be present in the population descriptive data. 


Given the timing of the start-up of this study and the ending of the ELMC grants for HS 
programs, with data collection occurring in the final months of the grant period, the project was 
also limited in the type of data it could collect. One result of the timing was that the study could 
not be designed to observe the coaching process over time (e.g., how the grantees got started, 
whether models were implemented with fidelity, or if the grantees implemented their coaching 


'* The telephone interviews were semi-structured and open-ended conversations; thus an item-by-item response rate, 
as calculated with survey data, can also be calculated with the interview qualitative data. Validity checks for the 
interview data were performed by comparing the audio recordings of the full telephone interviews to the interview 
data capture sheets and interview notes. Over 10 percent of the interviews were validity checked. For additional 
information on the validity checks and the interview data quality assurance procedures, see Volume 2: Appendixes, 
Appendix D. 

'’ The study was contracted about seven months after the ELMC grantees were awarded their funds and 
implemented their coaching programs. In addition, this study required Office of Management and Budget (OMB) 
approval, which was awarded in November 2011—over a year after OHS awarded the ELMC funds to grantees. 
Consequently, the project team had only four months to collect information about the ELMC initiative before the 
expected end date of the grants in February 2012 (excluding a few grantees with no-cost extensions to February 
2013). 
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models as they originally intended). As designed, the project also could not identify the impacts 
of coaching or any causal relationships among the features of coaching and other variables. Any 
associations or cross-tabulations presented in this report should not be interpreted as causal links. 
Furthermore, the subgroup analyses presented in this report are exploratory in nature, and they 
are subject to generalizability limitations related to the small cell counts. 


It is also important to recognize that all of the findings from this study come from self-reported, 
retrospective data. It is also important to remember that the outcomes reported herein are 
retrospective perceptions. Self-reported data may have validity problems because of the 
subjective nature of the information. For similar evaluation efforts in the future, it would be 
beneficial to collect other data, such as activity logs from the coaches, or direct observations of 
coaching sessions. Additional benefit could be derived from more extensive data collection (e.g., 
interviews) with grantee leadership and additional program staff. 
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Data Analysis Procedures 


The primary goal of data analysis for this study was to describe coaching among ELMC grantee 
recipients. For each research question, we analyzed both quantitative survey data and qualitative 
interview data. A variety of data were used to address each research question. 


The survey data analyses included univariate summary statistics for each variable presented in 
the report. For questions with fixed response categories, the univariate statistics included 
response frequencies and percentages. For questions with continuous or fixed-category numeric 
responses, univariate statistics included averages (measured using means or medians depending 
on the response type) and standard deviations (SDs) to indicate response variability. Graphical 
displays, including bar charts and pie charts, were also used to present descriptive data for some 
of the survey questions. '° 


Some bivariate descriptive analyses were also conducted for a subset of variables. Bivariate 
subgroup analyses (cross-tabulations with significance testing) were conducted to describe group 
differences in coaching, with subgroup sets defined for both grantees (according to grantee 
characteristics and contextual variables) and coaches (according to coaching features). We 
examined subgroup differences for seven grantee-level subgroup sets (grantee size, urbanicity, 
high or low concentration of dual-language learners, whether or not the grantee was AIAN and 
MSHS programs, providing EHS only or HS only or other program option, center based or not, 
and prior level of support for professional development), and also for four coach-level subgroup 
sets (type of staff coached, coach education and experience, caseload size, and coaching contact 
formats). Exhibit 8lists the bivariate subgroup analyses that were conducted for this study. '” 


'© Additional details about the survey analysis procedures are in Volume 2: Appendixes, Appendix E. Sample 
weights were not used in the survey data because they are not applicable for census, full-population-level data (i.e., 
the surveys were not of a sample). The qualitative analyses use thematic analysis approaches, in which sampling is 
not appropriate because the analysis is not intended to be representative of a population, to provide key themes 
related to context and personal perspectives of the ELMC initiative. 

" Detailed definitions of the subgroup sets are provided in Volume 2: Appendixes, Appendix E, along with results 
of key subgroup analyses including some bivariate correlations. Due to space constraints, some subgroup analyses 
were not included in this report. 


American Institutes for Research The Descriptive Study of the Early Learning Mentor Coach Grant Initiative—24 


Exhibit 8. Subgroup Analyses by Subgroup Set 


Subgroup Set Coaching Dependent Variable 


e Whether grantee offers coaching to any of 6 different staff types in their program 


Number of professional development supports grantee offered out of a total of 11 different 
supports 


Grantee challenges with hiring coaches 

Grantee challenges with coach turnover 

Months it took grantee to hire first coach 

Number of coaches hired from within grantee 

Grantee likelihood of ongoing coaching efforts after ELMC grant ends 

Grantee took any of 4 steps to sustain coaching 

e Whether grantee offers coaching to any of 6 different staff types in their program 
e Whether grantee used any of 7 national professional development (PD) resources 
e Whether grantee’s community had any of 8 local resources for TA and PD 
Grantee challenges with hiring coaches 

Grantee challenges with coach turnover 

Months it took grantee to hire first coach 

Number of coaches hired from within grantee 

Grantee likelihood of ongoing coaching efforts after ELMC grant ends 

Grantee took any of 4 steps to sustain coaching 


Grantee size 


Grantee urbanicity 


Grantee percent dual- 
language learners 


Grantee challenges with hiring coaches 
How much grantee prioritizes a language and culture match in hiring coaches 
Grantee goals related to cultural responsiveness and services for dual-language learners 


Grantee challenges with hiring coaches 
How much grantee prioritizes a language and culture match in hiring coaches 


AIAN and MSHS 


grantee type e Grantee goals related to cultural responsiveness and services for dual-language learners 
Eroor roomy e Whether grantee offers coaching to any of 6 staff types 
grantee type e Grantee goals related to cultural responsiveness and services for dual-language learners 


Whether grantee offers coaching to any of 6 staff types 
Number of professional development supports grantee offered out of a total of 11 
Grantee selection of any of 18 goals 

Frequency of 26 coaching strategies 

Frequency of coaching on 25 topics 

Coach identifies staff needs with any of 7 methods 
Frequency of coaching interactions not in person 
Frequency of 26 coaching strategies 

Frequency of adopting 10 roles in coaching work 
Whether coach orientation covered any of 7 topics 
Coach satisfaction with quantity of orientation 

Coach satisfaction with content of orientation 
Frequency of 26 coaching strategies 

Frequency of adopting 10 roles in coaching work 


Center-based only 
program option 


Coach type of staff 


Coach education & 
experience 


Coach caseload size 
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Subgroup Set Coaching Dependent Variable 


e Coach identifies staff needs with any of 7 methods 

e Extent to which the coach varies strategies, goals, and topics for different staff 
e Frequency of coaching interactions not in person 

e Frequency of 26 coaching strategies 

e Frequency of coaching on 25 topics 

e Frequency of adopting 10 roles in coaching work 

e Coach identifies staff needs with any of 7 methods 


Coach contact formats 


For the interview data and open-ended survey data, we conducted thematic analyses to identify and 
report patterns within the qualitative data. The thematic analyses on summaries of the interviews '® 
used a semantic approach, where themes were identified based on the explicit meanings of the data 
or what had explicitly been said in the interview data. This included approaches such as 
enumerating (e.g., counting categories), convergence (e.g., several categories were combined to 
represent a larger theme), divergence (e.g., large themes were separated into more specific ideas 
and concepts), and typology (e.g., classifying categories). Frequencies and counts of the thematic 
categories for the population and the subgroups are reported in combination with the survey data, 
and illustrative quotations from the interviews are included. ° 


'® As a result of budget and time constraints, full transcripts of the interviews were not analyzed. However, 
systematic summary data capture notes completed during and immediately after the interviews, which captured key 
data from the interviews, were used. 

'9 Additional details about interview analysis procedures are in Volume 2: Appendixes, Appendix D. 
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Study Findings 


The study findings are presented in seven sections corresponding to the key aspects of coaching 
and their related constructs examined in this study as described in Exhibit 4 (page 17): (1) the 
context of coaching, (2) basic dimensions of coaching, (3) structural dimensions of coaching, (4) 
procedural dimensions of coaching, (5) outputs of coaching, (6) perceived outcomes of coaching, 
successes and challenges, and (7) sustainability. 


Context of Coaching 


The community and history of a HS program may influence the ELMC implementation. For 
example, programs with substantial ongoing efforts may have found it easier to integrate the 
ELMC initiative; however, the ELMC initiative may have had the biggest impact on programs 
with relatively limited established professional development efforts. Other community 
characteristics may effect qualifications of coaches and availability of professional resources 
(e.g. rural program; lack of community colleges). 


Grantee Characteristics 


The grantees that responded to the grantee census survey had some diversity in terms of program 
type, program size, and other characteristics. Almost two-thirds of the grantee respondents (63.3 
percent) were combination HS/EHS grantees, whereas the majority of the other grantee 
respondents were either HS only (19.2 percent) or EHS only (10.8 percent). Exhibit 9 shows the 
characteristics of the grantee survey respondents in several categories, including program type, 
size, urbanicity, the percentage of DLLs, and program options, with percentages for each 
characteristic within these categories. As previously mentioned, we analyzed subgroup 
differences across these seven grantee-level subgroup sets (see Exhibit 8). 


Exhibit 9. Grantee Characteristics of Respondents to ELMC Grantee Census Survey 


Grantee Characteristics Percentage of Grantees 
Program Type 
HS and EHS 63.3% 
HS only 19.2% 
EHS only 10.8% 
American Indian and Alaskan Native (AIAN) 2.5% 
Migrant and Seasonal Head Start (MSHS) EHS 2.5% 
MSHS 1.7% 
100.0% 
Grantee Size 
Less than 400 funded children 35.8% 
400 to 1,000 funded children 28.3% 
1,001 to 5,000 funded children 35.0% 
More than 5,000 funded children 0.8% 
100.0% 
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Grantee Characteristics Percentage of Grantees 


Grantee Urbanicity 


Large urban (MSA with 1 million or more residents) 44.2% 

Small urban (MSA with less than 1 million residents) 29.2% 

Rural (not an MSA) 26.7% 
100.0% 

Percentage of DLLs 

Less than 50 percent DLLs 74.2% 

50 percent or more DLLs 25.8% 
100.0% 

Program Option 

Center-based only 25.8% 

Other program options 74.2% 
100.0% 


Note. N = 120. The source is the Head Start PIR administrative data set. PIR data are missing for one grantee 
survey respondent, so characteristic information is available for only 120 grantees. MSA indicates metropolitan 
Statistical area, as defined in the PIR. DLL indicates dual language learners. 


Grantee and Community Context 


The grantee and community context may be an important factor contributing to effective coaching. 
The grantees were asked on the survey about professional development community resources, their 
own requirements, and professional development supports for teachers and other staff. As shown in 
Exhibit 10, the ELMC grantees reported a high prevalence of using technical assistance and 
professional development resources in their local communities, and many of the grantees also 
reported using these resources for professional development, materials, or trainings. More than 
three-fourths of the grantees reported using community colleges or faculty (76.9 percent) and 
community mental health centers or mental health professionals (76.0 percent) as resources. More 
than two-thirds of the grantees reported using universities or faculty (71.1 percent), community 
service organizations (69.4 percent), early childhood resource and referral centers (67.8 percent), 
and libraries (66.9 percent). Fewer grantees reported using immigrant or cultural community 
organizations (40.5 percent) or other HS programs (38.0 percent), which were resources that that 
also tended to be less available in the communities. The subgroup analyses found that rural grantees 
were less likely than urban grantees to have access to immigrant or cultural community 
organizations, although there were no other differences in access to professional development 
supports between the rural and urban grantees.” 


The majority of the grantees also reported receiving professional development support from 
OHS’ national technical assistance centers. Among the grantee respondents, 63 percent reported 
they received professional development support from the National Center on Quality Teaching 
and Learning, 52 percent reported that they received support from the National Center on Parent, 
Family, and Community Engagement, and 32 percent reported that they received professional 
development support from the National Center on Cultural and Linguistic Responsiveness. 


*°An important note is that since the ELMC initiative was a competitive grant, HS grantees awarded ELMC funds 
may have had greater resources than the overall universe of HS grantees. 
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Exhibit 10. Type of Local Community Resources by Grantee and Grantee’s Use 


Usage Among 


Present in Grantees With 

Type of Resource Community Used by Grantee | Access 

Frequency | Percent | Frequency | Percent Percent 
Community colleges 107 88-43 | 9B 76.86 | 86.92 
Universities/faculty 106 876 86 71.07 | 81.13 
Other HS/EHS grantees | 69 57.02 | 46 38.02 66.67 
Resource and referral agencies 97S (80-7 82 67.77 84.54 
Community mental health centers/ professionals 111 91.74 92 76.03 82.88 
Other community service organizations 110 | 90.91 | 84 _ 69.42 | 76.36 
Immigrant/cultural community organizations 70 57.85 49 40.5 | 70.00 
Library | 108 | 8926 81 66.94 75.00 


Note. The sample size is 121 grantees. 


The grantees reported requiring a range of professional development hours for different staff 
members, as shown in Exhibit 11. All of the grantee respondents required annual professional 
development for classroom teachers, with the median being 20 hours per year per teacher. For 
other staff types (e.g., assistant teachers, home visitors, family child care staff, and other staff), 
the grantees ranged from having no professional development requirements to 41 or more hours 
of professional development per year. Not all of the grantees reported on all types of staff (e.g., 
not all grantees had home visitors or family child care staff). 


Exhibit 11. Range of Annual Professional Development Hours Required by ELMC Grantees, 
by Type of Staff 


Type of Staff | N | 15HoursorLess)| 16-30Hours | 31-40Hours| | 41 or More Hours 
Classroom teachers 118 43 41 5 29 
Assistant teachers 112 41 42 4 25 
Home visitors 92 39 26 2 25 
Family child care staff 49 32 10 1 6 
Other staff 77 34 24 1 18 


Note. The source is the grantee census survey. The grantee survey question was as follows: Approximately how many hours of professional 
development are required per year for the following staff types? The response options included integers from 0 to 40 or a categorical response of 
41 hours or more. Not all of the grantees had all staff types. 


The majority of the staff that we interviewed (86 percent) said they were interested in pursuing 
additional classes or professional development opportunities. Nearly all of the grantees had 
ongoing efforts to help teachers obtain a child development associate (CDA) credential, an 
associate’s degree, or a bachelor’s degree. All of the grantee survey respondents also reported 
that they provided supports or resources to encourage staff to pursue professional development, 
training, and education. 


As shown in Exhibit 12, most of the grantees offered tuition reimbursement support, substitute 


teachers to cover classrooms, and printed or electronic materials and resources. On average, the 
grantees offered 7.1 types of supports or resources, out of the 11 types listed in Exhibit 12. 
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Exhibit 12. Supports and Resources Provided by the Grantees for Staff Professional Development, 
Training, and Education 


Type of Professional Development, Training, or Education Resources Number Yes Percentage Yes 
Grantee helps staff obtain CDA credentials, AA degrees, or BA degrees@ 118 99.0% 
Tuition reimbursement 104 86.0% 
Substitute teachers to cover classrooms 95 78.5% 
Printed or electronic materials and resources 91 75.2% 
Paid release time 79 65.3% 
Purchasing course textbooks 77 63.6% 
Flexible schedules 71 58.7% 
Transportation reimbursement 50 41.3% 
Unpaid release time 48 39.7% 
Continuing education units (CEUs) 47 38.8% 
Provide associate’s and bachelor’s degree courses on-site 28 23.2% 


Note. The source is the grantee census survey. The sample size is 121 grantees. All of the grantees reported providing supports or resources to encourage staff 
to pursue professional development, training, and education; the grantees were then asked to select all that apply from the list in the table. 


@The sample size for this question is 119 grantees. 


It is interesting to note that less than one-half of all the grantees (44.6 percent) offered coaching 
as a professional development resource prior to receiving the ELMC grant. Among the grantees 
that had coaching before the ELMC grant, most (75.9 percent) indicated that the ELMC grant 
was used to supplement, not replace, prior efforts. 


The subgroup analyses found no differences between small (serving up to 400 children) and 
large grantees (serving 401 children or more) in the number of professional development 
supports they offered (Exhibit 12). Similarly, there are no significant differences in the number 
of professional development supports offered by center-based-only grantees and grantees that are 
not only center-based.~' 


As illustrated in Exhibit 13 the grantees are fairly evenly divided into four categories of support 
for professional development and coaching prior to the ELMC grant. Among the grantees, 

24.0 percent had high professional development supports (i.e., offering 8 or more of the 

11 professional development supports; Exhibit 12) and had a coach before the ELMC grant; 24.0 
percent had high professional development supports but did not have a coach before the ELMC 
grant; 20.7 percent had fewer professional development supports (offering fewer than 8 of the 11 
professional development supports) but had a coach before the ELMC grant; and 31.4 percent 
had fewer professional development supports and did not have a coach before the ELMC grant. 


*! Additional information on this and other subgroup analyses referenced in this report is in Volume 2: Appendixes, 
Appendix E. 
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Exhibit 13. The Percentages of Grantees With Different Levels of Prior Support 
for Professional Development and Coaching 


Fewer PD supports, 


High PD supports, No coach before 
Had coach before 


High PD supports, 
No coach before 


Fewer PD supports, 
Had coach before 


Note. The source is the grantee census survey. The sample size is 121 grantees. Professional Development (PD) supports refers to resources such as 
tuition support, substitute teachers, onsite classes for staff development. High PD= More than 8 resources offered. 


Summary of Context of Coaching Findings 


The study findings indicate that the grantees have diverse characteristics, except that the majority 
of grantees were combined HS/EHS programs. Virtually all of the grantees provided professional 
development—particularly for teachers—and many supported their staff in obtaining further 
credentials, providing supports to do so. The ELMC initiative fit within an ongoing professional 
development context. 


Basic Dimensions 


The first aspects of coaching are the basics—namely, administrative decisions that need to be 
made regarding the goals of coaching, whom to coach, whom to hire as coaches, and how long to 
provide coaching. These decisions define the foundations of a coaching program, and could 
influence staff attitudes, progress monitoring and overall effectiveness. For example, hiring 
coaches who are not experienced and qualified ECE professionals may undermine staff 
perceptions of the coaching, and in turn reduce their level of engagement. 


Coaching Goals 
Grantee-level goals may shape the content of coaching sessions, and could also inform decisions 
regarding monitoring of staff-level, service-level, and program-level coaching outcomes. When 


asked about their goals and objectives, nearly all of the grantees (96.7 percent) reported that they 
had identified overarching grantee-level goals and objectives for the ELMC grant. As shown in 
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Exhibit 14, the most common goals endorsed were improving classroom quality, including the 
quality of staff practices with children (87.1 percent), and improving CLASS assessment scores 
(72.4 percent).*” Many of the grantees also reported goals related to child learning and 
development, including improving child outcomes (61.2 percent) and training teachers on 
teaching school readiness topics (54.3 percent). 


Exhibit 14. Grantee Goals for the ELMC Initiative: Percentage of Grantees Selecting Each Goal 
When Asked to Identify Up to Five Overarching Goals for the Coaches 


Grantee Goals Percentage Selected 


Commonly Endorsed Goals 

Improve the quality of staff practice with children 87.1% 
Improve CLASS assessment scores 72.4% 
Improve assessed child outcomes 61.2% 
Train on teaching of school-readiness topics 54.3% 
Less Frequently Endorsed Goals 

Train on a particular curriculum 25.9% 
Train on behavior management 25.0% 
Improve established coaching 23.3% 
Support for education and career development 23.3% 
Improve services for dual-language learners 19.8% 
Improve other assessment scores (e.g., Early 19.8% 
Childhood Environment Rating Scale) 

Support for using assessments 18.1% 
Improve the quality of staff practice with families 14.7% 
Support administrative staff and/or supervisors 12.1% 
Rarely Endorsed Goals 

Support the use of new technology 9.5% 
Improve cultural responsiveness 6.9% 
Improve service for children with disabilities 6.0% 
Improve parent engagement 5.2% 
Other 5.2% 


Note. The source is the grantee census survey. The sample size is 116 grantees. Grantees could select up to 
5 goals from a list of 18 options. 


Overall, relatively few of the grantees (19.8 percent) identified improving services for dual- 
language learners (DLLs) as a goal, and very few of the grantees (6.9 percent) indicated that 
improving cultural responsiveness was a goal. In the subgroup analyses, programs serving high 
concentrations of DLLs were not more likely than other grantees to endorse either of these 
goals. 


*° The grantees were asked to select up to five goals from the list in Exhibit 14. The average number of goals 
selected was 4.92, which indicated that nearly all of the grantees selected 5 goals. 
°° Additional details are in Volume 2: Appendixes, Appendix E. 
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Staffing of Coaches 


Grantees were awarded ELMC initiative funds in September 2010, and the majority of the 
grantee administrator respondents (72.1 percent) hired their first coaches by December 2010. The 
remainder of the grantees (27.9 percent) hired their first coaches in 2011, most by February. 
Exhibit 15 shows the month in which the grantees reported hiring their first coaches for the 
ELMC grant. One reason that many of the grantees were able to hire their coaches fairly quickly 
after being awarded the grant is that many of the grantees (76.2 percent) reported using existing 
staff as coaches; that is, they hired coaches who had already worked for the grantee, mostly in 


jobs other than coaching. 


Exhibit 15. Month When Grantees Hired Their First Coaches 


25.0% - 
20.7% 
19.8% 
20.0% - 18.9% 18.9% 
15.0% - 
12.6% 
10.0% - 
5.0% - 3.6% 3.6% 
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Note. The source is the grantee census survey. The sample size is 111 grantees. Months in which no coach was hired are not listed in the chart. The ELMC 


grant began in October 2010. 


At the time the data were collected, the total number of ELMC coaches working at a grantee 
ranged from 1 to 8, with an average of 2.9 (SD = 1.7). 
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Exhibit 16. Grantee Number of Part-Time and Full-Time Coaches Funded by ELMC Grants 


Number of Funded Number of Funded Full-Time Coaches 
Part-Time Coaches 0 ey Re) Total 
0 5 40 13 58 

1-3 17 14 2 33 

4-8 18 3 1 22 

Total 40 57 16 113 


Note. The source is the grantee census survey. The sample size is 113 grantees. 
The shaded cells indicated the twenty grantees that funded a combination of full-time and 
part-time staff with the ELMC grant. 


Grantees used ELMC funding to pay for a complex blend of full-time and part-time coach 
positions, as shown in Exhibit 16. About one-third (35 of 113) of the grantee respondents 
reported that they fund only part-time coaches, with the number of part-time coaches ranging 
from 1 to 8. Approximately half (55 of 113) of the grantee respondents reported that they fund 
only full-time coaches, with the number of full-time coaches ranging from 1 to 5. Twenty 
grantees (shaded cell in Exhibit 16) used ELMC funds to pay for a combination of full- and part- 
time coach positions. Five grantees reported that they did not use ELMC funds to pay for any 
full-time or part-time coaches; of these, one used existing Head Start Curriculum Specialists as 
coaches, two reported that they used coaches from outside agencies, and the other two reported 
using “mentors.” None of the five explained fully why these staff did not count as full-time or 
part-time coaches paid for by the ELMC grant. However, some grantees may not consider 
coaches to be regular full-time or part-time staff, as data from the coach survey indicated that 
one-half of the coaches (49.9 percent) funded by the ELMC initiative were hired as external 
consultants or temporary employees, not as permanent HS staff. 


Exhibit 17 shows that although relatively few of the ELMC coaches hired (19.6 percent) were 
already coaches at the grantee, many of the coaches hired (57.1 percent) were currently working 
for the grantee in some other non-coaching capacity or had previously worked for the grantee in 
some non-coaching capacity (39.4 percent). 
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Exhibit 17. Percentages of Grantees That Hired at Least One Coach With Previous 
or Current Experience With the Grantee 


T T 


Currently worked for grantee as a Currently worked for grantee in Previously worked for grantee in 
coach some other capacity some capacity 


Note. The source is the grantee census survey. Across the three questions represented in this graph, the sample size ranged from 105 to 
112 grantees. Three questions are represented in this graph: Of all the coaches that you have hired with ELMC funds since September 2010, 
(1) how many were already working for your grantee as a coach? (2) How many were already working for your grantee in some other 
capacity? (3) How many had worked previously for your grantee in some capacity? The respondents could answer any number from 0 to 40 
or could check a box to indicate 40 or more. 


Coach Background and Qualifications 


The grantees reported the qualifications required for their coaches. The largest portion of the 
grantees required that the coaches have a minimum of a bachelor’s degree (85 percent), with 
about 40 percent requiring a bachelor’s degree in addition to ECE specialization. As shown in 
Exhibit 18 and Exhibit 19, the majority of the coaches were white (75.8 percent) and highly 
educated. Ninety percent had at least a bachelor’s degree and more than 45 percent had a 
graduate-level degree (Exhibit 19). The majority of the coaches (85.2 percent) also held 
additional relevant certifications or credentials, such as a teaching certification ora CDA 
credential. Half of the coaches had substantial experience in ECE (i.e., 50 percent with 18 or 
more years of ECE experience), and half had at least 12 years of experience providing 
professional development, including teaching, training, or facilitating groups of adults. About 
20.7 percent had less than five years of experience and 8.4 percent had one year or less. 
However, many of the coaches did not have extensive experience in actually being coaches 
before the ELMC initiative. Fifty percent of the coaches reported having two years or less of 
experience as coaches before being hired for the ELMC initiative. 
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Exhibit 18. The Racial and Ethnic Composition of the Coaches 


Ethnicity 

Hispanic or Hispanic or Non-Hispanic Non-Hispanic 

Latino (n) Latino (%) Co) a EV Haten (7) or Latino (%) 
White 24 6.4% 258 69.4% 
Black or African American 3 0.8% 66 17.7% 
Asian 1 0.3% 10 2.7% 
AIAN 3 0.8% 7 1.9% 
Native Hawaiian or Other Pacific Islander 0 0.0% 0 0.0% 


Note. The source is the coach census survey. The sample size is 372 coaches. 
Exhibit 19. The Education Levels of the Coaches 
Doctorate or 


professional degree 
4.2% 


Graduate coursework 
13.1% 


Note. The source is the coach census survey. The sample size is 382 coaches. The doctorate and professional degree category includes doctoral 
degrees (such as Ph.D. or Ed.D.) and professional degrees (such as M.D., D.D.S., M.B.A., J.D., and L.L.B.). The master’s category includes master’s 
degrees (such as M.A. or M.S.); the graduate coursework category includes graduate or professional coursework with no degree; the bachelor’s category 
includes bachelor’s degrees (such as B.A. or B.S.); the associate’s category includes an associate's degree (A.A.); and the some college category 


includes some college but no degree. 


In the survey, the grantee administrators were asked to rank the importance of five specific 
qualifications for hiring coaches. Exhibit 20 shows the percentages of grantees that ranked each 
qualification as often or always important. In hiring coaches, nearly all of the grantees (98.2 
percent) selected interpersonal skills as the qualification that was often or always important in 
their hiring. After interpersonal skills, many of the grantees also responded that experience in 
training, teaching, mentoring, or coaching adults was a qualification that was often or always 
important in their hiring decisions. A minority of the grantees (37.7 percent) indicated that 
familiarity with the HS center/staff or program was an important qualification. 
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Exhibit 20. The Percentages of Grantees That Identified Qualifications of the Coaches as Often 
Important or Always Important in Hiring Decisions 


400.0% 98.2% 
U7/O > 


88.8% 
— 79.1% 
iti 68.5% 
60.0% - 
0, 
ose! 37.7% 
0.0% | 


Interpersonal skills | Experience Experience with = Language and Familiarity with 
training, teaching, HS/EHS programs culture match with __ center/staff or 
mentoring, or staff and/or families program or worked 
coaching adults and children there previously 


Note. The source is the grantee census survey. The total sample size ranges from 111 to 116 grantees. The grantees were asked the following question: “What 
other qualifications were important to your hiring decisions for the early learning mentor coaches?” The response options included always important/necessary, 
often important, sometimes important, never important/not necessary, or don’t know. 


Exhibit 21 illustrates their responses about the most important qualifications for successful 
coaching. Coaches were asked to select the three most important qualifications; they were most 
likely to report that interpersonal skills (68.4 percent) were among the three. A background in 
ECE and child care (46.7 percent) and experience training, teaching, mentoring, or coaching 
adults (38.6 percent) were also considered important by many of the coaches. 


Exhibit 21. Percentages of Coaches Who Selected Each Item as Among Top Three 
Most Important Qualifications for a Coach to Be Successful 


Important Qualifications of the Coach Percentage 
Interpersonal skills (€.g., ability to establish relationships) 68.4% 
Background in ECE and child care 46.7% 
Experience training, teaching, mentoring, or coaching adults 38.6% 
Degree in ECE or a related field 28.5% 
Ability to provide constructive feedback 28.2% 
Background in teaching 19.1% 
Experience with reflective practice or supervision 19.1% 
Experience with HS programs 18.5% 
Knowledgeable about adult learning strategies and principles 10.2% 
Time management skills 6.0% 
Familiarity with center/staff or program (e.g., worked there previously) 4.2% 


American Institutes for Research The Descriptive Study of the Early Learning Mentor Coach Grant Initiative—37 


Important Qualifications of the Coach Percentage 


Background in working with families 3.1% 
Background in clinical work (e.g., counseling) 3.1% 
Language and culture match (with staff and/or families and children) 18% 
Experience with home visitors 13% 
Background in management work (e.g., administration) 0.3% 


Note. The source is the coach census survey. The sample size is 383 coaches. The coaches were asked to select the top three 
qualifications that were most important for them to be successful. 


The grantee administrators (in the survey) and coaches (in the telephone interviews) also 
provided open-ended responses about the qualifications of coaches most important to the success 
of coaching. Consistent with the coach survey, both grantees (Exhibit 22) and coaches (Exhibit 
23) in their interview most often noted interpersonal skills as an important qualification. The 
interpersonal skills listed by the grantees included the ability to build trusting relationships, being 
able to listen, respecting others, and being good at building rapport. The key types of 
interpersonal skills the coaches described were trust and communicating with staff (Exhibit 23). 


Exhibit 22. Qualifications of Coaches Most Important for Success: Grantees’ Survey 


Important Qualifications of the Coach Percentage 


Personality and interpersonal skills 41.7% 
Knowledge 19.2% 
Expertise and practical skills 16.6% 
Experience and background 15.8% 
Education and qualifications 6.8% 


Note. The source is the grantee census survey. One hundred nineteen grantees responded to the 
following question: “List up to three qualifications of the coach that were most important for the 
success of coaching at your grantee.” There were 148 open-ended responses because the grantees 
could provide more than one qualification. 


Exhibit 23. Types of Interpersonal Skills Most Important for Successful Coaching as Described by 
the Coaches 


Successful Interpersonal Skills | n Percentage | 
Trust 24 53% 
Communicating/getting to know staff 20 44% 
Listening 9 20% 
Flexible 7 16% 
Respectful 5 11% 

Keep confidentiality 3 71% 


Note. The source is the coach telephone interview. Forty-five coaches reported data related to this theme; 
more than one answer was possible. There is duplicate counting because 23 respondents gave more than 
one answer. 


Staff Targeted for Coaching 
Both the surveys and the interviews were used to gather information about how the centers and the 


staff were selected to receive coaching support. Although almost two-thirds of the grantees (63.3 
percent) reported that all of their centers were receiving coaching, the grantees also reported using a 
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variety of methods to determine which centers would receive coaching. 


Many coaches reported that they worked with a variety of staff types. Only about one-fourth of 
the coaches (24.1 percent) worked with a single staff type only. Coaches reported working with 
an average of 2.3 staff types, which could include any combination of lead teachers, assistant 
teachers, home visitors, family child care staff, administrators, supervisors, other administrators, 
or other. Many of the coaches (42.0 percent) worked with 2 to 3 different staff types; 16.2 
percent worked with 4 to 6 different staff types; and 0.5 percent worked with 7 different staff 


types. 


The most common type of staff coached was HS classroom staff, including lead teachers (77.7 
percent) and assistant teachers (58.5 percent). Many of the coaches also reported working with 
EHS classroom staff, including teachers (33.3 percent) and assistant teachers (24.9 percent). 
Some of the coaches reported working with home visitors (19.4 percent), supervisors (17.1 
percent), administrators (11.5 percent), and other staff types (15.0 percent). Only 3.9 percent of 
the coaches reported working with family child care staff. Grantees respondents and coach 
respondents reported similar patterns regarding the types of staff coached. 


Over one-half of the coaches (56.7 percent) reported working only with classroom staff, either 
with lead teachers only or with both lead and assistant teachers (Exhibit 24). Nearly one-fourth 
of the coaches (23.9 percent) reported working with other combinations of classroom staff, 
including classroom staff, administrators, supervisors, family child care staff, and other staff 
types, but excluding home visitors. About one-fifth of the coaches (19.4 percent) reported 
working with home visitors, mostly along with classroom staff and sometimes with other 
program staff.** The remaining 23.9 percent of the coaches reported working with other 
combinations of classroom staff, including administrators, supervisors, family child care staff, 
and other staff types, but excluding home visitors. The subgroup analysis found no differences in 
the type of staff coaches served by grantee size or urbanicity.”° 


** Fighty-nine of the grantees offered EHS programs, mostly in combination with HS, but a few were EHS only. 
Only 31 of the grantees (26 percent) were center-based only. 
°° Additional details are in Volume 2: Appendixes, Appendix E. 
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Exhibit 24. The Percentages of Coaches Who Worked With Different Combinations of Staff 


All Other Staff Type Lead Teachers Only 


Combinations = -~_ 18.9% 


23.9% 


Home Visitors plus ; 
Other Staff Lead and Assistant 
19.4% Teachers Only 
. 37.8% 


Note. The source is the coach census survey. The sample size is 381 coaches. 
Summary of Basic Dimensions Findings 


When asked about their goals and objectives, nearly all of the grantees (96.7 percent) reported 
that they had identified overarching grantee-level goals and objectives for the ELMC grant. The 
most common goals endorsed were improving classroom quality. Overall, very few of the 
grantees indicated that improving cultural responsiveness was a goal, and relatively few of the 
grantees identified improving service for DLLs as a goal. 


We found that the majority of HS grantees were able to hire coaches quickly after being awarded 
ELMC grants, with most hiring full-time coaches or a combination of part- and full-time coaches 
within the first three months of receiving the grant. The most important qualification for hiring 
identified by grantee administrators was interpersonal skills. Grantees, coaches and staff all 
agreed that interpersonal skills were important to the success of the coach. Coaches were highly 
educated, and 50 percent had 18 years or more of ECE experience. 


Nearly two-thirds of the grantees reported that all of their centers (76 of 120 grantees) were 
receiving coaching through the ELMC grant. If not every center was selected, the grantees 
reported using a variety of methods to determine which centers would receive coaching through 
the ELMC grant. Most of the staff members who were coached were classroom staff. More than 
one-half of the coaches reported working only with classroom staff, either with lead teachers 
only or both lead and assistant teachers. Nearly two-thirds of the grantees reported that all of 
their centers (76 of 120 grantees) were receiving coaching through the ELMC grant. If not every 
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center was selected, the grantees reported using a variety of methods to determine which centers 
would receive coaching through the ELMC grant. Most of the staff members who were coached 
were classroom staff. More than one-half of the coaches reported working only with classroom 
staff, either with lead teachers only or both lead and assistant teachers. 


Structural Dimensions 


With an understanding of the larger context in which coaching takes place and the basic aspects 
of coaching, another aspect that is important for describing coaching programs is the structure of 
their initiatives. Structural aspects include as training and supervision of coaches, and coach 
workload such as hours worked, number of programs served and coach-to-staff ratio. These 
features have potential consequences for coaching process and oucomes. For example, it is 
possible that coaches who receive training specific to the grantee goals and objectives could 
provide even more effective and consistent professional development. 


Training of Coaches 


As illustrated in Exhibit 25, the majority of the grantees (87.4 percent) reported that they 
provided a formal orientation and ongoing training for their coaches. A majority of the coaches 
concurred that they received both initial orientation and ongoing training as a part of the ELMC 
initiative. 


Exhibit 25. Percentages of Grantees and Coaches Reporting Orientation and Ongoing Training 


Grantees that Coaches that Grantees that Coaches that 
reported providing reported receiving reported providing reported receiving 
an orientation an orientation ongoing training ongoing training 


Note. The source is the grantee census survey and the coach census survey. The sample size ranges from 119 to 120 for the 
grantee responses and from 381 to 384 for the coach responses. The response options were yes or no; this exhibit reports the 
proportion of yeses. 


Of the coaches who received an orientation or initial training, their survey responses indicated 


that 87.4 percent were at least moderately satisfied with the amount of training that they 
received. Similarly, of the coaches who received ongoing training, 88.7 percent were at least 
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moderately satisfied with the content of the training they received. Only a very small number of 
the coaches (2.9 percent) reported that they were not at all satisfied with the amount of training 

and could have used more, and only 2.3 percent of the coaches were not at all satisfied with the 

content. 


In the telephone interviews, we asked the coaches to describe in detail the type of training or 
orientation they received from their grantees (Exhibit 26). Of the 50 coaching interview 
respondents, few (16 percent) described receiving training related to coaching. The most 
common theme related to training that emerged from the interviews was about self-directed 
training, (e.g., looking for resources online or independently reading resources that they thought 
could help their work with staff). 


Exhibit 26. Types of Training the Coaches Received 


Types of Training | n || Percentage 
Coach did not receive formal training/self-trained 17 34% 
Someone trained coach on assessments 15 30% 
Someone trained coach on grantee/programmatic information 10 20% 
Someone trained coach on coaching and/or people relationship skills 8 16% 


Note. The source is the coach telephone interview. Fifty coaches reported data related to this theme. 
Supervision of Coaches 


On the survey, most of the grantee administrators (89.9 percent) reported that coaches had 
supervisors assigned to them. Among the grantees that had supervisors for their coaches, most 
(86.9 percent) reported that the coaches met regularly with the supervisors, whereas the remainder 
(13.1 percent) did not regularly meet with the supervisors. 


In the coach telephone interviews, 48 coaches (88.8 percent) provided additional details about who 
supervised them (Exhibit 27). In the interviews, the coaches most often described being supervised 
by the HS director, followed by the education coordinator or manager. Three coaches indicated 
that a specialist or a lead coach supervised them. The type of supervision the coaches described 
varied in terms of the purpose of the supervision (e.g., sharing general information or specific 
details about staff), the frequency of meeting with the supervisor (e.g., weekly, monthly, or as 
needed), and the form of supervision (e.g., individually or in groups with other coaches). 
Supervision approaches were very individualized for each grantee. For example, one coach 
explained that she was supervised by the HS director, whom she described as being very 
supportive and as always having time to support the coach program. She met with her supervisor 
once per week. They typically discussed the management of meetings she had with the staff she 
coached, and she kept the supervisor up to date on her work. Another coach that was supervised by 
the ELMC grant manager reported that she met with her supervisor about once a month to discuss 
coaching strategies, challenges, and positive things that were happening, and to assess the ELMC 
initiative in terms of what data were being collected, what to change, and what was important in 
their coaching model. This coach described her supervisor as serving many roles, including 
emotional support, technical support, and as a source of encouragement. 
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Exhibit 27. Types of Coach Supervisors 


Coach Supervisors n Percentage 
HS director 21 44% 
Education coordinator/manager 11 23% 
ELMC grant manager 7 15% 
Specialist/lead coach or outside agency 6 13% 
No one/minimal supervision 4 80% 
this theme. 
Coach Workload 


The grantee survey respondents reported using ELMC funding to provide coaching support for 
between 1 and 27 of their centers, with an average of 9.3 centers per grantee (Standard 
Deviation: SD) = 6.7) *° being served by the ELMC initiative. On average, the coaches reported 
that they were working in 4.4 centers (range = 0 to 28; SD = 4.4), although a few respondents 
serving a large number of centers skew the average. Most of the coaches we interviewed (n = 27; 
79 percent) found providing coaching in multiple locations challenging (see page 70 for more 
details about the perceived challenges of working in multiple locations). Two types of challenges 
emerged from our interviews: (1) The coaches felt that they did not get to see the staff enough, 
and (2) the coaches felt that they did not have enough time to work with the staff. (We report on 
this and other perceived challenges in greater detail starting on page 67). Three survey 
respondents reported not working in any centers; two of these coaches worked only with home 
visitors, and the third reported only helping other coaches. Among coaches working in centers, 
more than one-third (34.7 percent) reported working in five or more centers. Exhibit 28 presents 
data from the 366 coaches who reported working in at least one center.” 


Exhibit 28. Proportion of Coaches Serving a Specific Number of Centers 


1 center 26.5% 
2-4 centers 38.8% 
5-9 centers 24.9% 
10-19 centers 8.7% 
20-28 centers 1.1% 
Total 100% 


Note. The source is the coach census survey. The sample size is 366 coaches. The participants could 
select any numerical value as a response. 


As shown in Exhibit 29, the most common staff types the coaches reported working with were HS 


°° The standard deviation (SD) represents how much variation or dispersion there is among the individual data 
values of study participants from the overall group average or mean value. It is a measurement of the spread of 
individual data points around the mean of all the data. 

*” Three respondents reported that they do not work in center-based programs; it may be that some of the 15 mentor 
coaches who did not respond to this question also work in programs that are not center based. 
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lead teachers (n = 296) and assistant teachers (n = 223). Coaches who reported working with lead 
and assistant teachers in HS centers reported a median caseload of 6 inclusive of these two staff 
types together. Coaches also reported working with EHS lead teachers (n = 127) and assistant 
teachers (n = 95). Coaches working with these staff types in EHS programs reported a median 
caseload of 3 for both EHS lead teachers and EHS assistant teachers. A smaller group of coaches 
also worked with home visitors (n = 74), supervisors (n = 65), administrators (n = 44), and family 
child care staff (n = 15), with caseloads for those staff types ranging from 2 to 4. 


Most of the coaches reported working together with a combination of staff, or staff teams, at 
least some of the time, with 43.2 percent reporting that they met with teams of staff frequently, 
36.3 percent reporting that they did so sometimes, 8.7 percent reporting that they seldom did so, 
and only 11.8 percent reporting they never met with teams of staff. 


Exhibit 29. Percentage of Mentor Coaches Working With Each Staff Type 
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Note. The source is the coach census survey. The sample size is 381 coaches. 


Coaches reported working a median of 25 hours per week as a coach for the ELMC grantee. 
Although only 21.4 percent (82 of 383) of the coaches reported they held another job position 
with the grantee in addition to their coaching position, as Exhibit 30 indicates, 43.9 percent of 
the coaches were involved in doing non-coaching work. The hours worked per week by the 
coaches in non-coaching work ranged considerably. More than one-quarter of the coaches (27.5 
percent) reported spending 20 or more hours per week on non-coaching work, and 44 percent of 
the coaches (163 of 371) reported spending at least some time each week doing work for the 
grantee that was not part of their coaching position. 
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Exhibit 30. Proportion of Coaches Reporting Coaching and Non-Coaching Hours Paid by the ELMC 
Grant per Week 


Paid as a Coach Hours per Week in Non-coaching Work 
Hours per Week 
(EY )) (ey A) 
0 hours@ 3.7% 56.1% 
1-19 hours 31.5% 16.4% 
20-39 hours 28.8% 19.1% 
40 hours 35.2% 6.5% 
41 hours or more 0.8% 1.9% 


Note. The source is the coach census survey. The sample size ranges from 371 to 375 coaches. 
aCoaches who reported being paid 0 hours per week as a coach had other positions at the grantee, except for one coach who worked 0 hours for the 
grantee but reported being funded by another local agency and taking on a caseload of staff at the ELMC grantee. 


Summary of Structural Dimensions Findings 


The majority of the ELMC grantees reported providing a formal training orientation to their 
coaches, and nearly all of the grantees reported providing ongoing training to their coaches. Most 
of the grantees reported that they have staff assigned to supervise the coaches. On average, the 
coaches reported that they were working in about 4 centers. The most common staff types the 
coaches reported working with were HS lead teachers and assistant teachers. Coaches who 
reported working with lead and assistant teachers in HS centers reported a median caseload of 6 
inclusive of these two staff types together. Coaches worked a median of 25 hours per week as a 
coach for the grantee, but the range of the hours worked per week by the coaches was 
considerable. 


Procedural Dimensions 


Procedural dimensions include components such as identifying staff needs (and outlining action 
steps aligned with these needs), establishing staff goals, providing information and resources, 
modeling and practicing, engaging in observation and reflection, and providing feedback on 
practices and behavior (Snyder et al., 2012). Any given coaching session could vary markedly 
across these features, depending on both staff need and coaches’ skills. 


Identifying Staff Needs 


Coaches reported seven methods for identifying staff needs to be addressed in the coaching 
session. First, in the coach survey, nearly all of the respondents (97.4 percent) reported that the 
staff they worked with self-identified what they wanted to work on in the coaching session. Most 
of the coaches also reported that they identified staff needs by observing staff with both formal 
observational assessment tools (85.4 percent) and informal assessment tools (78.9 percent). 
Other common sources of information included child assessment data (51.8 percent) and 
performance reviews by staff supervisors (49.5 percent). Fewer of the coaches reported that the 
grantee administration identified the targets for all of the staff (22.7 percent) (e.g., improvement 
of classroom factors associated with CLASS; or a specific curriculum supplement), and there 
was also less use of HS monitoring reviews (19.0 percent) to identify the staff needs for 
coaching. In the telephone interviews with staff members who received coaching, we asked how 
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they were selected to receive coaching. The most common responses the staff provided were that 
either all staff participated at their center or they did not know why they were selected to receive 
coaching. 


Subgroup analyses compared coach use of the seven methods to identify staff needs that are 
listed in the previous paragraph. The subgroup analyses of the survey data found that the 
coaches’ methods of identifying staff needs varied by the type of staff they worked with, the type 
of interaction formats they frequently used, and the size of their coaching caseload. Coaches with 
larger caseloads used more resources to identify staff needs, particularly data results such as 
child assessments, classroom observations, and monitoring review results.*? Coaches with fewer 
interactions with staff and those working only with lead teachers tended to use fewer methods of 
identifying staff needs.~” 


Establishing Staff Goals 


Coaches in the survey noted that improving staff skills and strategies was the most common 
goals for staff, as shown in Exhibit 31. 


Exhibit 31. Most Common Goals That the Coaches Targeted With Staff 


Goals Frequency Percentage 


Improve staff skills and strategies 195 34% 
Program/classroom operations 117 21% 
Improve use of assessment or technology 103 18% 
Increase staff professional knowledge 46 8% 
Meeting national/program standards 38 7% 
Encourage staff personal growth 34 6% 
Improve structure/organization 30 5% 
Cultural competency 6 1% 


Note. The source is the coach census survey. The sample size is 350 coaches. The coaches were asked to identify up to three common 
goals for the targeted work with staff. Thus, the coaches could provide more than one answer. 


In the telephone interviews, we asked both the coaches and the staff to describe in detail 
their main coaching goals and topics. Exhibit 32 categorizes the goals as described by the 
coaches in the interviews, and Exhibit 33 categorizes the goals as described by staff in the 
interviews. 


Exhibit 32. Goals the Coaches Described in Their Interviews 


Goals n | Percentage 
Improving teacher and staff quality 23 45% 
Improving CLASS assessment scores and training 14 26% 
Providing general support to staff 10 19% 
Developing a sustainable coaching model 9 17% 


*8 Additional details are in Volume 2: Appendixes, Appendix E, Table E-38. 
** Additional details are in Volume 2: Appendixes, Appendix E, Tables E-24 and E-29. 
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Improving teaching of a school-readiness topic (e.g., literacy and mathematics) 6 11% 
Supporting continuing education and the professional development of staff 6 11% 
Supporting and improving the classroom environment 6 11% 
Improving assessed child outcomes 5 9% 
Improving work with families 5 9% 


Note. The source is the coach telephone interview. Fifty-four coaches reported data on this theme. However, there is duplicate counting 
because 24 individuals reported more than one answer. 


Exhibit 33. Goals the Staff Described in Their Interviews 


Goals n Percentage 
Improving the physical environment of the classroom 23 28% 
Improving teacher quality in the classroom 21 26% 
Improving teaching of a school-readiness topic (e.g., literacy and mathematics) 17 21% 
Implement CLASS and improve CLASS assessment scores 16 20% 
Providing behavior management techniques 16 20% 
Providing activities and materials for the classroom 14 17% 
Improving classroom management and child engagement 12 15% 
Providing support around an individual child 10 12% 
Providing support and training around a curriculum 9 11% 


Note. The source is the staff telephone interview. Eighty-one staff members reported data on this theme. However, there is duplicate counting 
because 45 individuals reported more than one answer. 


Staff members also shared in the telephone interview their perspectives about goals and topics 
they covered with their coaches. Staff largely talked about the goals of improving the classroom 
environment, teacher quality in the classroom, and their CLASS scores. For example, many staff 
described working with their coaches to improve the quality of their teaching (e.g., the coach is 
involved in promoting the verbal communication skills of the staff with open-ended questions, 
language modeling, and promoting higher-order thinking in the children). One staff member said 
that she now notices what she is doing when working with the children; everything is much more 
intentional than before. For CLASS, the staff reported that the coaches would help them 
understand the process of CLASS or work through improving their skills in multiple domains to 
improve their scores. 


Based on the interview data, the coaches and the staff were generally aligned in their 
understanding of the focus and the goals of the coaching initiative. However, it is important to 
note where the points of view of the two populations were slightly different. Where a staff 
person may have seen the coach focus on the classroom environment and management, the coach 
may have framed the same work as an effort to improve CLASS scores or teacher quality and 
skills. Differences may have resulted because of different questions asked in each interview. 
Although the coach was asked in the interview about the grantee’s goals for the ELMC initiative, 
the teacher was asked specifically about the goals and topics of their particular coaching work 
on-the-ground. Staff members were also prompted to provide examples about their work with the 
coaches, which led to more specific and concrete topics than coaches’ responses, which focused 
on the overarching plan for the initiative. 
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Topical Areas in Coaching Sessions 


In the survey, the coaches reported covering several different topical areas in their work, as 
shown in Exhibit 34. Nearly all of the coaches reported frequently covering at least one topic 
within each of the following categories: increasing or improving staff skills and strategies (93.0 
percent) and increasing staff professional knowledge (91.6 percent). Many of the coaches also 
reported frequently covering at least one topic in these categories: encouraging staff personal 
growth (76.2 percent), improving structure and organization (70.6 percent), and improving the 
use of assessment and technology (58.2 percent). Each of these categories included multiple 
subtopics, which were also strongly endorsed.” 


Exhibit 34. Content of Coaching Frequently Covered With the Staff by the Coaches 


100.0% 93.0% 


80.0% - 


60.0% - 


40.0% - 


20.0% ~ 
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Staff skills and Professional Personal growth = Structureand § Assessment and 
strategies knowledge organization technology 


Note. The source is the coach census survey. The sample size ranges from 378 to 383. The response categories included frequently, 
occasionally, hardly ever, or never. 


Staff skills and strategies included eight topics in the survey: teacher-child interactions, staff use 
of language with children, staff responsiveness to children, instructional practices for specific 
developmental domains, the implementation of specific curricula, encouraging parent-child 
interactions, engaging parents, and an “other” topic. The coaches reported frequently covering a 
median number of four topics related to staff skills and strategies (range = 0—7). The most 
frequently covered topics in this category included teacher—child interactions (84.3 percent), staff 
use of language with children (81.3 percent), and staff responsiveness to children (77.7 percent). 


Professional knowledge included seven topics in the survey: developmental domains (e.g., 
literacy and socioemotional), behavior management, CLASS scores, the needs of children with 


°° A validity check of the data did not reveal cases where the respondents selected only one response for every 
subtopic in every category. 
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identified disabilities or other special needs, the needs of DLL children, the needs of culturally 
diverse families; and an “other” topic. The coaches reported frequently covering a median 
number of three topics related to professional knowledge (range = 0-7). The most frequently 
covered topics in this category included developmental domains (76.7 percent), behavior 
management (66.3 percent), and CLASS scores (55.2 percent). 


Personal growth included five topics in the survey: self-efficacy, motivation, and empowerment; 
positive interactions with colleagues; enrollment in professional development; enrollment in 
college coursework; and an “other” topic. The coaches reported frequently covering a median 
number of two topics related to personal growth (range = 0-5). The most frequently covered 
topics in this category included self-efficacy, motivation, and empowerment (62.3 percent) and 
positive interactions with colleagues (60.1 percent). 


Structure and organization included six topics in the survey: the use of books and other 
educational materials; classroom or center organization; the use or display of materials; home 
organization, management, and safety; content and organization of the home visit; and an “other” 
topic. The coaches reported frequently covering a median number of two topics related to 
structure and organization (range = 0-6). The most frequently covered topic in this category was 
the use of books and other educational materials (59.0 percent). 


Assessment and technology included four topics in the survey: ongoing child assessment for 
tailoring instruction, ongoing child assessment for program quality improvement, the overall use 
of technology, and an “other” topic. The coaches reported frequently covering a median number 
of one topic related to personal growth (range = 0-4). The most frequently covered topic in this 
category was ongoing child assessment for tailoring instruction (50.3 percent). 


The subgroup analyses compared coach coverage of 25 specific topics in the five topic areas 
shown in Exhibit 34, and found that the content covered by the coaches varied according to the 
type of staff coached and the frequency of in-person interactions. Specifically, coaches who 
worked with lead teachers tended to cover a narrower range of topics, and coaches working with 
home visitors were less likely to cover certain classroom-focused topics such as CLASS scores.”! 
Also, coaches who reported fewer face-to-face interactions with staff also tended to report less 
coverage of a number of coaching topics, including the needs of children with special needs, the 
implementation of curricula, improving structure and organization, improving the use of 
technology and assessment, and positive interactions with colleagues.” 


Interacting with Staff 


As shown in Exhibit 35, the coaches reported using a variety of formats to interact with the staff, 
including both in-person, phone, and electronic. Nearly all of the coaches (98.1 percent) in the 
survey reported having individual face-to-face meetings with each staff at least monthly, in 
addition to other interaction formats. Of the seven coaches who reported never having individual 
face-to-face meetings with the staff, two reported holding group face-to-face meetings and 


3! Additional details are in Volume 2: Appendixes, Appendix E, Exhibit E-28. 
*° Additional details are in Volume 2: Appendixes, Appendix E, Exhibit E-35. 
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exchanging e-mails, whereas the other five reported no group face-to-face meetings but used a 
variety of other interaction formats, including phone, e-mail, video exchanges, and other 
messaging interactions. 


Exhibit 35. Frequency of Coach Interactions With Typical Staff, by Interaction Format 


Less Than Once 


Once per Week or More 


Interaction Formats 


per Week 
Phone call 358 24.0% 44.4% 31.6% 
Individual face-to-face meeting 367 1.9% 41.1% 56.9% 
Group face-to-face meeting 355 12.1% 59.2% 28.7% 
E-mail 358 18.2% 34.1% 47.8% 
Online messaging 339 91.4% 3.8% 4.7% 
Texting 342 70.5% 15.2% 14.3% 
Virtual meeting 341 93.0% 4.1% 2.9% 
Social media 340 95.6% 3.2% 1.2% 
Video camera 345 50.7% 41.4% 7.8% 


Note. The source is the coach census survey. The sample size ranges from 339 to 367 coaches. The rating options included never, about once per 
month, about every other week, about once per week, about once a day, or more than once a day in a typical month. The categories were condensed in 


this table. 


The subgroup analyses shows that the interaction formats used by the coaches varied according 
to the type of staff coached but not according to the coach caseload size.” 


Exhibit 36 groups the coaches into four categories of interaction frequency with the staff they 
coached. For this exhibit, high in-person interaction corresponds with meeting face-to-face once 
per week or more in a group or individually; high remote interaction involves interacting 
remotely at least once per week through phone calls, e-mail, or other online formats. Of the 
coaches, 38.1 percent were in the high in-person high remote interaction category; 21.1 percent 
were in the high in-person low remote interaction category (meeting face-to-face at least once 
per week but interacting remotely less than once per week); 18.6 percent were in the low in- 
person high remote interaction category (meeting face-to-face less than once per week but 
interacting remotely at least once per week); and 22.2 percent were in the low in-person low 
remote interaction category (meeting face-to-face less than once per week and interacting 
remotely less than once per week). 


°° Additional details are in Volume 2: Appendixes, Appendix E. 
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Exhibit 36. Percentage of Coaches With Different Levels of Interaction Frequency 
With Staff 


Lowin-person 
interactions, low 
remote interactions 
22.2% 


High in-person 
interactions, high 
remote interactions 
38.1% 


Low in-person 
interactions, high 
remote interactions 

18.6% 


High in-person 
interactions, low 
remote interactions 
21.1% 


Note. The source is the coach census survey. The sample size is 365 coaches. High in-person high remote interaction category = meeting face-to-face at 
least once per week in a group or individually; high remote interaction involves interacting remotely at least once per week through phone calls, e-mail, or 

other online formats. High in-person low remote interaction category = meeting face-to-face at least once per week but interacting remotely less than once 
per week. Low in-person high remote interaction category = meeting face-to-face less than once per week but interacting remotely at least once per week. 
Low in-person low remote interaction category = meeting face-to-face less than once per week and interacting remotely less than once per week. 


Roles of Coaches 


As shown in Exhibit 37, the coaches were asked in the survey about their perceptions of their 
most frequent roles. Most of the coaches reported a role of collaborative partner with the staff 
they supported (74.5 percent), followed by the roles of emotional supporter (58.3 percent), and 
teacher/instructor for adults (58.2 percent). The coaches reported taking on an average of 4 of the 
10 suggested roles (SD = 2.22) asked about in the survey, with responses ranging from 0-10 
roles. Relatively few of the coaches (17.5 percent) reported that they frequently took on a 
supervisory role with the staff they coached. 
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Exhibit 37. Roles Frequently Taken on by the Coaches 
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Note. The source is the coach census survey. The sample size is 384 coaches. The response categories included frequently, occasionally, 
rarely, or never. 


In the telephone interviews, the staff members were asked to describe the types of roles they saw 
their coaches taking with them. They were probed on the roles of assistant, advocate, emotional 
support, and crisis intervention, and were allowed to suggest others. The four roles identified 
through the interview probes comprised the bulk of the staff responses, although there were other 
roles that came up with some frequency, as detailed in Exhibit 38. 


Exhibit 38. Staff Perspectives About the Frequent Roles of Coaches 


Role n | Percentage 


Emotional support 47 58.8% 
Assistant in the classroom 37 46.3% 
Teacher to staff 35 44.8% 
Advocate in the HS program 25 31.3% 
Crisis intervention 11 13.8% 
Leader 7 8.8% 
Peer 6 7.5% 


Note. The source is the staff telephone interview. Eighty staff members 
reported data related to this theme. However, 53 respondents reported more 
than one component. 
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Emotional Support. In the staff interviews, nearly 60 percent of the staff described their coaches 
as an emotional support. One respondent said “encouragement, encouragement, and 
encouragement” was what the coach provided, such as when the teacher was upset by how her 
classroom was operating and the coach talked to her and gave her a hug. Other staff members 
similarly felt that their coaches were an emotional support when they had an issue, professional 
or personal, or when they just wanted someone to talk with about their work. One home visitor 
expressed her gratitude for the emotional support she received from her coach: “The coach was a 
listening ear to some of the challenges that we [as home visitors] faced based on that lack of 
emotional support.” 


Assistant in the Classroom. The staff (46.3 percent) also described that their coaches acted as an 
assistant in the classroom, to a much greater extent than coaches in the survey described that 
role. In the telephone interviews, the staff explained that the coaches were often people who 
came into the classroom and actively worked with the children. Some staff members provided 
examples of particular moments, such as when a child misbehaved during dismissal time and the 
coach stepped in to help; other staff members described coaching as a more open approach, such 
as one teacher who said the coach would “join us in circle time; that’s just the teacher in her.” 


The subgroup analyses compared the frequency with which coaches adopted the 10 roles listed in 
Exhibit 37, and found that the roles assumed by the coaches varied according to the type of 
interaction formats frequently used by the coaches, the coach education and experience level, 
and the coach caseload size. Coaches with fewer interactions with staff tended to take on some 
roles less often. Coaches with more experience or education were less likely to take on certain 
roles outside the scope of coaching, such as assistant, emotional supporter, or crisis intervener. 
Coaches with larger caseloads more frequently reported taking on several roles, including 
emotional supporter, logistical supporter, and teacher or instructor for adults. *4 


As a Formal Supervisor. From the coach survey, About one-fifth of the ELMC coaches (21.5 
percent) responded that they were formal supervisors of program staff. However, only a small 
proportion (14.2 percent) of the coaches also coached staff members that they supervised. In the 
interview, the majority of coaches (n = 36; 77 percent) responded that they were not formal 
supervisors to the staff they coached. However, a large majority of coaches did report to somebody 
else at the grantee (i.e., program director, manager of teachers) about the progress that the staff 
were making in coaching. In the interviews, 40 percent of the coaches explained they provided 
staff-specific information to the supervisors, and 20 percent said they provided general 
information to the supervisors (Exhibit 39). For example, one coach explained that when she 
completed a classroom evaluation of a teacher, she would discuss the specific results with the 
supervisor of the teacher before showing the results to the teacher. Another coach explained that 
she was not required to report specific information about a teacher to a supervisor, and she 
believed that the staff needed to feel that their coaching sessions were confidential. As such, this 
coach reported information only to the supervisor in broad terms about the topics she was 


** Additional details are in Volume 2: Appendixes, Appendix E, Exhibit E-33, Exhibit E-37 and Exhibit E-40. 
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working on with the staff and did not provide specific details. 


Exhibit 39. Coaches Reporting Information to Staff Supervisors 


Coach Supervision of Staff n Percentage 
Coach reports staff information to staff supervisor about specific staff progress. 18 40% 
Coach reports staff information to staff supervisor about staff in general. 7 16% 
Coach reports staff information to own supervisor about staff in general or with 
specifics. 16 36% 
Coach reports staff information to no one. 4 9% 


Note. The source is the coach telephone interview. Forty-five of the 59 telephone Interviewees addressed this theme. 


The reporting of information by the coach to the supervisor, as shown in Exhibit 39, is similar to 
the information collected from the staff about reports of their coaching work. In the telephone 
interviews, the staff members were asked if their coaches reported to their supervisors, and, if so, 
whether they felt it interfered with the coach-staff relationship. Fifty percent of the interview 
sample (40 out of 80 respondents) said their coaches reported to their supervisors, and 35 percent 
said the coach did not report information to their supervisors. Of the 40 staff members who were 
supervised by their coaches, only 3 felt that this supervision interfered with coaching. The 
majority was either positive (n = 21) or neutral (n = 16) about the effect of the coach’s 
supervisory activities on the coaching process. Some respondents felt that communicating with 
supervision facilitated coaching because the coach was able to build communication channels 
with the administration to ensure that they were “on the same page.” For example, one staff gave 
an example in the interview that the coach talking to the administration helped her because it 
meant the coach was fully informed about requirements and the policies of the program (e.g., 
that teachers cannot use food in an art project) and the center director and other supervisors were 
kept informed of the classroom and staff needs. 


Coaching Strategies 


In the survey, the coaches were asked to report their strategies, including subsets of observation, 
feedback, and discussion strategies; practice and modeling strategies; and support strategies. The 
coaches also reported on their frequency of strategy use, on average, with a typical staff member 
that they were coaching. Overall, looking across all the strategy categories in the survey, the 
coaches were most likely to report using the following strategies at least once per month with a 
typical staff member: introducing new skills, practices, or strategies (98.7 percent); reflecting on 
skills, practices, or strategies (98.2 percent); providing verbal feedback based on live 
observations (95.8 percent); setting and reassessing goals for individuals (95.7 percent); and 
conducting live, on-site observation (95.0 percent). 


Observation, feedback and discussion related types of strategies, coaches were most likely to 
report using on-site observation, verbal feedback, and reflection strategies with each staff person 
at least three times in a typical month, whereas few of the coaches reported using peer 
observation or video strategies. The observation, feedback, and discussion strategies that were 
most commonly used at least three times per month with an individual staff member included the 
following: providing verbal feedback based on live observations (54.5 percent); reflecting on 
skills, practices, or strategies (52.5 percent); providing verbal feedback based on discussion with 
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staff (50.0 percent); conducting live, on-site observation (with or without an observation tool, 
47.8 percent); introducing new skills, practices, or strategies (43.2 percent). 


Practice and modeling strategies were less frequently used than observation, feedback and 
discussion types of strategies, although many of coaches still reported using them (n = 361-379) 
(Exhibit 40). The most common modeling strategy endorsed was demonstration or modeling in 
the work setting; 40.6 percent of the coaches reported using this strategy three or more times per 
month; 52.5 percent of the coaches reported using it one or two times per month, and only 6.9 
percent of the coaches reported that they never used this strategy. 


Exhibit 40. Coaches Reporting Use of Practice and Modeling Strategies with Staff in a Typical 
Month 


1-2 Times 3 or More Times 


Practice and Modeling Strategies 


per Month per Month 
Demonstrate or model skills and strategies while in work setting 379 6.9% 52.5% 40.6% 
Ask staff to practice skill and report back 379 7.7% 62.0% 30.3% 
Coach “on the fly” 362 | 37.3% 39.5% 23.2% 
Work with staff to role-play a skill or a strategy 362 | 30.1% 52.2% 17.7% 
Demonstrate or model skills and strategies while not in work setting | 361 | 38.8% 45.2% 16.1% 


Note. The source is the coach census survey. The sample size ranges from 361 to 379 coaches. The rating options included never, 1-2 times, 3-4 times, and 
more than 4 times in a typical month. The 3-4 times and more than 4 times categories were combined in the table. 


Supportive types of strategies were also asked about in the survey (e.g., problem solving with the 
staff on personal issues; providing emotional support; working on stress reduction; sharing 
materials and resources; conducting or arranging an on-site workshop or training; helping with 
preparation, administration, and scheduling; and working as an assistant in the classroom, on a 
home visit, or in the child care room). Sharing materials and resources was the supportive 
strategy that the coaches most commonly reported using three or more times per month (60.0 
percent) with their individual staff. Generally speaking, however, most supportive strategies 
were used at least once a month with the typical staff. Most of the coaches reported providing 
emotional support (96.8 percent), working on stress reduction (89.2 percent), and problem 
solving with staff on a personal issue (78.8 percent) at least once a month. 


Variation of Strategies Used. Coaches were asked how often they varied their strategies, 
depending on which staff they were working with. In the survey, less than 13 percent (12.7 
percent) reported that their strategies for coaching were almost always consistent across the staff; 
36.9 percent reported that their strategies were more consistent than varying, 38.3 percent 
reported that their strategies were more varied than consistent, and 12.1 percent reported that 
their strategies were almost always varied. 


Subgroup analyses compared coach use of 26 specific strategies within the three broader 
categories of observation, feedback, and discussion strategies; practice and modeling strategies; 
and support strategies. The subgroup analyses found that the strategies used by coaches varied 
according to the type of staff coached and the type of interaction formats frequently used by the 
coaches, but not by the coach education and experience level or caseload size. Specifically, 
coaches who worked only with lead teachers were less likely to use a number of coaching 
strategies, including facilitating group discussions, conducting on-site trainings, and modeling 
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skills. Coaches working with home visitors were somewhat more likely to use certain strategies, 
particularly conducting on-site trainings and providing feedback by e-mail, text, or online, as 
shown in Volume 2. Coaches with more frequent face-to-face interactions with staff tended to 
report more use of observation, feedback, and discussion strategies and of practice and modeling 
strategies in their work. Coaches with few interactions with staff, either face-to-face or 
otherwise, tended to report little use of supportive strategies.” 


We asked an open-ended survey question about which strategies the coaches felt were most 
successful in working with the staff. The respondents could select up to three types of strategies. 
Exhibit 41 outlines the frequency of the strategies selected. Interestingly, although modeling 
strategies were used less frequently than observation, feedback and discussion, the coaches most 
frequently cited modeling as among the most effective strategies used. 


Exhibit 41. Coach Reporting of the Most Effective Strategies for Changing Staff Practices 


Strategies Frequency Percentage | 
Modeling, demonstration, or role-play 234 65% 
Provide materials, resources, or information 197 55% 
Meetings, discussion, or problem solving 140 39% 
Observation 107 30% 
Reflection 92 26% 
Using video, audio, or photos 71 20% 
Feedback 67 19% 
Training 36 10% 
Coteaching or collaborating 37 10% 
Assessment tools and guides 20 6% 
Relationship building 19 5% 
Developing and tracking goals 17 5% 
Practice 8 2% 


Note. The source is the coach census survey. The sample size is 360 coaches. The respondents could pick up to 
three categories. 


Summary of Procedural Dimensions Findings 


Coaches reported a variety of approaches for identifying staff needs. Nearly all of the coaches 
reported that the staff self-identified their needs for coaching. Most of the coaches also reported 
that they identified staff needs by observation, and formal and informal assessment. The coaches 
reported in the surveys that the most common goals they targeted were improvement in the 
general skills and strategies and knowledge of the staff. All the coaches reported covering 
several different topical areas in their work. 


Most coaches were not formal supervisors of the staff they coached, but many reported providing 
specific information to a staff person’s supervisor. Most of the coaches perceived their roles as 


*° Additional details are in Volume 2: Appendixes, Appendix E, Exhibit E-26 and Exhibit E-29. 
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being collaborative partners with the staff they supported. 


The coaches were most likely to report using on-site observation, verbal feedback, and reflection 
strategies with each staff person at least three times in a typical month, whereas few of the 
coaches reported using peer observation or video strategies. The coaches reported using a variety 
of format modes to interact with the staff, which indicates that coaching occurred both in-person 
and through phone and electronic formats. 


Outputs of Coaching 


A key aspect of coaching is the quality of the coach-staff relationship and the level of staff 
openness to and engagement in coaching. Coaches must first develop rapport with the staff 
members if they are to gain their trust and respect, and staff members must be open and engage 
in the coaching process. For this to happen a level of trust must be developed and the staff must 
be willing to improve and be “coachable” (i.e., able to self-reflect, share mistakes, make use of 
coach feedback). 


Staff Openness To and Engagement in Coaching 


In the telephone interviews, we asked the coaches to describe the level of openness and 
engagement for selected staff who would also complete interviews. On average, the coaches 
rated their coachees as an 8.1 on a 10-point scale in terms of the level of engagement and 
openness (where 10 = most engaged and open). Coaches were asked to explain their ratings. 
Exhibit 42 presents the themes that emerged from the coaches’ explanations of ratings about staff 
openness and engagement. 


Exhibit 42. Coach Ratings of the Level of Openness and Engagement Among the Staff 


Theme n Percentage 
Positive attitude about being coached 39 38% 
Open to hearing coach’s suggestions 30 29% 
Actively participates in coaching sessions 26 25% 
Shows effort to implement changes 18 18% 
Engagement increased over time 14 14% 
Visible changes to classroom 13 13% 


Note. The source is the coach telephone interview. One hundred two coaches reported data 
related to this theme. However, there is duplicate counting because 45 individuals reported more 
than one component. 


In their discussions of working with individual staff members, the coaches most commonly noted 
a positive attitude about being coached from staff, using terms such as “receptive,” 
“enthusiastic,” and “excited.” For example, one coach explained that a staff member who was 
concemed about limited time said that “she was going to make it work one way or another.” 
Another common theme was that the staff was open to hearing coaches’ suggestions. For 
example, a coach described a staff person as being “excited to get outside ideas” on how to 
improve in areas where she was not as strong as she would like to be. The coaches noted that the 
staff members would take an active role in the coaching sessions by asking questions, reflecting, 
raising classroom issues, or actively problem-solving. The teachers and the other staff were not 
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always onboard with coaching from the beginning, but the coaches reported that some of these 
initially resistant staff became more engaged over time. The coaches also explained that it took 
time to demonstrate to staff that they were coming in to be part of the program team. A few of 
the coaches were more concrete in explaining their ratings of staff openness and engagement, 
basing their ratings on the resulting visible changes in the classroom, such as improvements in 
behavior management and nurturing relationships. 


Coach-Staff Relationship 


We also asked the staff in the telephone interviews about their perceptions of the relationships 
they had with their coaches. Interviewers asked staff, “How would you describe your 
relationship with your mentor-coach?” followed by the optional probe, if deemed necessary by 
the interviewer: “Would you say your working relationship with your mentor-coach is 
comfortable and easy or sometimes challenging? Please explain what it is like to work with 
him/her.” The responses we received from the staff members tended toward the positive, and 
they generally stated that their relationships were good. For example, one staff member felt that 
she had a very positive and close relationship with the coach. The coach was always available 
and would have the teacher discuss her ideas and questions first before bringing up her own 
points. The coach shared ideas but did not force a solution. 


Three staff members provided negative reports on their relationship with their coaches. In all 
three cases, these staff members did not trust their coaches and did not feel comfortable sharing 
work mistakes with them. They referred to coaches as acting like police officers or supervisors 
(even though they were not supervisors). These negative views seem to partly come from 
perceptions of the coaches’ inexperience and inability to give positive or constructive feedback. 


To get more specific information about staff perceptions of the relationships with their coaches, 
we asked: “Overall, is your coach skilled and knowledgeable in areas helpful to you? If yes, 
please provide an example.” 


Exhibit 43 lists the major themes that categorize the staff perceptions of coaches’ skills and 
knowledge. The most common response was appreciation for the coach’s experience as a 
classroom teacher. For example, one staff member stated that her coach was a former teacher and 
therefore understood the classroom and “has been around the classroom long enough to 
understand when they [children] are acting up.” Another staff member shared that her coach “has 
so much experience that she can really give honest ideas about how best to meet the needs of the 
kids.” Another staff member summarized the perceptions of many by saying that “it was great 
having a coach who has been in a classroom environment.” 
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Exhibit 43. Staff Perceptions About Helpful Areas of Coach Skills or Knowledge 


Area of Skill/Knowledge n Percentage | 
Past experience as a classroom teacher 24 30% 
Specific instructional domains or curricula 15 19% 
Activities and materials for classroom 13 16% 
Classroom/behavior management 10 13% 
Problem-solving with individual children 10 13% 
Observation and assessment tools 5 6% 
Developmentally appropriate practices 5 6% 
Lesson planning 5 6% 
The coach has no helpful skills or knowledge 1 1% 


Note. The source is the staff telephone interview. Eighty staff members reported data related to 
this theme. However, there is duplicate counting because 21 individuals reported more than one 
component. 


Summary of Output of Coaching Findings 


Most of the coaches reported success in increasing staff openness to learning and improving the 
quality of practices. The staff reported changes in both instructional and behavioral management 
practices. Both the coaches and the staff reported that their relationships were supportive and 
open. 


Perceived Outcomes of Coaching, Successes and Challenges 


Although outcomes could not be directly assessed or evaluated within the design of this study, 
stakeholder perception of coaching outcomes can be examined. In addition to outputs of 
coaching, it is important to examine perceive coaching outcomes: what helps a coaching program 
to be successful, and what aspects of a coaching program may be challenging? 


Perceived Success and Changes as a Result of Coaching 


Overall, as shown in Exhibit 44, the grantee administrators strongly endorsed that coaching led 
to perceived successful changes with the staff. The grantees reported being most enthusiastic 
about the coach success in training and supporting staff and increasing staff openness to learning. 
Nearly all of the grantees reported at least moderate success in improving the quality of staff 
practices. 
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Exhibit 44. Percentages of Grantees Reporting That Coaches Achieved Large or Moderate Levels of 
Success in Different Areas 


400.0% 100.0% 
VO > 


80.0% - 
60.0% - 


40.0% + 


20.0% - it 
0.0% + 


Percentage reporting Percentage reporting Percentage reporting Percentage reporting Percentage reporting 
coaches were very or coaches led to large coaches made large coaches led tolarge coaches led to large 
moderately or moderate increase or moderate or moderate increase or moderate increase 
successful at training instaffopennessto improvementsin in use of assessment in career 
and supporting staff leaming quality of staff development of staff 
practice 


Note. The source is the grantee census survey. The sample size is 119 grantees. 


Staff Improvement. In the survey, the majority of the grantees (72.3 percent) reported that the 
coaches were very successful at training and supporting the staff they worked with, and the rest 
of the grantees (27.7 percent) reported that the coaches were moderately successful. None of the 
grantees reported that the coaches had limited success or were only somewhat successful. The 
grantees also reported high levels of success in increasing the openness to learning. The majority 
of the grantees (67.2 percent) reported large increases in the openness to learning, and the rest of 
the grantees reported a moderate increase (31.1 percent) or some increase (1.7 percent). 


Quality of Practices. All of the grantees reported that the coaches made at least some 
improvement in the quality of practices of the staff they coached. The grantees were fairly evenly 
divided between reporting large improvements (46.2 percent) and moderate improvements (49.6 
percent), whereas a handful of the grantees reported small improvements (4.2 percent). 


Use of Assessment. There was wider variability in the survey results in the grantee rating of 
coach success at improving and increasing the use of assessment by staff. Nearly all of the 
grantees reported that the coaches worked on assessment (95.2 percent), yet only 58 percent of 
mentor coaches said that assessment was a specific content area covered in their sessions with 
staff (see Exhibit 34 above). Most rated the coaches as having a large (36.1 percent) or a 
moderate (41.2 percent) increase in success at improving and increasing the use of assessment by 
staff they mentor—coached. A minority of the grantees indicated some increase (16.0 percent) or 
limited increase (2.5 percent). 
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Personal Growth. The grantees were less likely to rate a high level of coach success in increasing 
career development and the pursuit of education and training among staff. The majority of the 
grantees reported that their coaches worked on career development (93.3 percent), with 16.8 
percent indicating a large increase and 45.4 percent indicating a moderate increase in this work. 
A fair number of the grantees reported some increase (20.2 percent) or limited increase (10.9 
percent) in career development and the pursuit of education and training. 


As shown in Exhibit 45, a small percentage of the grantees reported that the coaches exceeded 
the grantee’s overarching grantee-level goals for the ELMC initiative (13.9 percent), while the 
majority reported that the coaches made substantial progress in meeting the goals 

(73.9 percent). Few grantees reported that the coaches made only some progress (12.2 percent), 
and none of the grantees reported that the coaches made no progress. 


Exhibit 45. Grantee Ratings of Coach Success at Meeting Grantee-Level Goals Among Grantees 
That Reported Goals for the ELMC Initiative 


Exceeded 13.9% 
i ———___ Made some 


progress 12.2% 


Made substantial 
progress 73.9% 


Note. The source is the grantee census survey. The sample size is 115 grantees. A few of the grantees (3.4 percent) reported 
that the grantee-level goals were not applicable. We do not know why grantees selected non applicable. 


As shown in Exhibit 46, most of the coaches were positive in their survey ratings of their own 
overall success for the ELMC initiative, with 42.1 percent reporting they were very successful 
and 46.1 percent reporting they were moderately successful. However, some of the coaches were 
less positive, with 10.5 percent reporting that they were somewhat successful, and 1.3 percent 
reporting limited success. 
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Exhibit 46. Coach Ratings of Their Overall Success for the ELMC Initiative 


Limited success 


1.3% Somewhat 


successful 10.5% 


Very successful 
42.1%, 


Moderately 
successful 46.1% 


Note. The source is the coach census survey. The sample size is 380 coaches. 


As shown Exhibit 47, the coaches were also fairly positive about their success in specific aspects 
of their work with the staff. Similar to the grantee reports, most of the coaches reported success 
with many or all of the staff in increasing openness to learning and improving the quality of 
practices. To note, the coach survey respondents appeared to be more modest in their 
endorsement of success, compared with the grantee survey respondents. 
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Exhibit 47. Percentages of Coaches Reporting Success in Their Work for the ELMC Initiative 


Many staff 
BAIl staff 


Staff more open to learning Staff improved quality of | Staff increased appropriate use 
practice of assessment 


Note. The source is the coach census survey. The sample size ranges from 381 to 383 coaches. 


Staff Attitudes. The coach ratings of their own successes working with staff followed a similar 
pattern as the grantee survey responses. The coaches were positive about their success at 
increasing openness to learning in the staff, with 26.9 percent reporting that all of the staff 
increased openness and 58.2 percent reporting that many of the staff did. The coaches were also 
positive about their success at improving the quality of staff practices. About one-fifth of the 
coaches (21.5 percent) reported that all of the staff improved, and 55.0 percent reported that 
many of the staff improved. Similar to the grantee responses, a somewhat smaller percentage of 
the coaches reported strong success in increasing the appropriate use of assessment, with 44.1 
percent reporting that many of the staff increased their use of assessment, and 17.5 percent 
reporting that all did. 


Personal Growth. About one-fifth of the coaches (19.7 percent) reported they did not work on 
increasing staff focus on career development and the pursuit of education and training. Among 
the 306 coaches who did work with staff on this issue, only 11.1 percent reported limited 
success, whereas 26.8 percent reported they were somewhat successful, 37.3 percent reported 
they were moderately successful, and 24.8 percent reported they were very successful. 


In the telephone interviews, we asked both the coaches and the staff to describe the changes that 
resulted from coaching. The themes that emerged from the coach interviews about the 
effectiveness of their coaching with staff are listed in Exhibit 48. The coach responses are based 
on individual staff, so the coaches could say that, for example, they were effective with staff 
member A but not effective with staff member B. 
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Exhibit 48. Coach Report of Effectiveness of Coaching for Individual Staff 


Goals n Percentage 
Coach gave only positive responses about effectiveness 69 66% 
Coach gave mixed responses about effectiveness 21 20% 
Coach could not provide an answer about effectiveness 8 8% 
Coach gave only negative responses about effectiveness 4 4% 
Note. The source is the coach telephone interview. One hundred and two coaches reported data on this theme. There is no 


duplicate counting. 


The coaches shared only positive indications of their effectiveness for the majority (66 percent) 
of the selected staff members discussed. For example, one coach said that she saw a “dramatic 
change” in one staff’s teaching styles and found her “willing to open up, take new approaches, 
and adopt new ideas.” Another coach reported that one of the staff she worked with received a 
promotion, gained knowledge and confidence, and pursued professional development outside the 
center. For 20 percent of the staff discussed, the coaches gave mixed responses about their 
effectiveness. These coaches acknowledged that there were some changes and indicators of 
effectiveness, but they also shared examples in which their work was not effective. For example, 
one coach said that although her staff person made many gains, she often went “three steps 
forward and one step back” because the staff person was overworked. Another coach felt that she 
did not have the maximum impact because of the limited number of sessions she had with the 
staff person, but the coach was able to see the staff teaching practice change over time. Overall, 
however, the coaches reported seeing some degree of effectiveness with 86 percent of the 
selected staff. 


In the interviews, we then asked the coaches how they measured their effectiveness with the 
staff. Exhibit 49 lists the themes that emerged from the interviews. Most coaches (60 percent) 
reported judging their effectiveness by their own observations of the staff or through 
conversations they had with the staff. For instance, one coach noted that she could see a change 
in a teacher’s videos as the term progressed. Other coaches said that they saw their staff using the 
techniques they discussed in coaching, such as displaying examples of child diction in the room, 
keeping children engaged when reading a story, and rearranging the schedule of the classroom to 
prevent behavior issues. Sometimes the coaches could see the progress of coaching through 
conversations they had with staff. One coach reported how a staff person was able to describe 
dimensions important to classroom quality. Standardized methods for tracking progress, such as 
formalized tracking and formal classroom assessments, were mentioned 17 percent of the time. 


Exhibit 49. Coach-Reported Indicators of Effectiveness 


Tafel (ers ice)g n Percentage 
Observations of staff work 36 40% 
Conversations between the coach and the staff 18 20% 
Engagement in and openness of coaching process 16 18% 
Feedback from staff on the coaching process 8 9% 
Tracked coaching goals 8 9% 
Classroom assessments 7 8% 


Note. The source is the coach telephone interview. Eighty-eight staff members made some positive reports about coaching 
effectiveness. However, there is duplicate counting because 17 coaches reported more than one indicator. 
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In the interviews, we asked the coaches to think across their work as coaches—and all the staff 
they worked with during the initiative—and share the areas where they saw the largest 
improvements. The emergent themes are listed in Exhibit 50. The coaches were most likely to 
describe that their work improved the ways that the staff interacted with the children. For 
instance, one coach said, “Now the teachers do a lot more talking and interacting with the 
children than they did prior to the coaching.” The coaches also observed an improvement in 
instructional practices (n = 12) as a result of the coaching. These responses were specific to 
improvements in the tools of instruction (e.g., specific activities or resources that were bolstered 
by coaching [n = 4]) and the approach of instruction (e.g., intentional practices and an 
understanding of developmental principles [n = 8]). For example, one coach referred to the more 
appropriate use of music in the classroom, while another reported helping the teachers learn how 
to make materials themselves instead of buying them. Regarding instructional approach, some 
coaches reported that the teachers were more intentional and aware of the importance of 
providing meaningful activities and interactions for the children. 


Exhibit 50. Coach Perceptions About the Improvements in Staff as a Result of Coaching 


Perceptions n Percentage 
Improvement in work with children, such as more intentional conversations 18 33% 
Improvement in instructional practices 12 22% 
Improvement in personal outlook on work 8 15% 
Improvement in classroom environment 6 11% 
Improvement in behavior management abilities 5 9% 
Improvement in work with staff, such as closer relationships 5 9% 


Note. The source is the coach telephone interview. Fifty-four coaches reported data on this theme. However, there is duplicate counting 
because 14 coaches reported more than one answer. 


In the staff interviews, we asked the staff to provide a rating of how effective they felt coaching 
was for them and how their work changed as a result of coaching. On average, the staff rated 
coaching effectiveness at 8.3 on a 10-point scale of effectiveness. The staff found coaching to be 
overwhelmingly positive and effective, with 86 percent of the staff giving a rating of 7 or higher. 
Seventy-seven percent of the staff would like to continue coaching. In addition, 20 percent of the 
staff reported that their coaches provided assistance or emotional support in their professional 
development efforts. 


Exhibit 51 categorizes the explanations that the staff members provided for their ratings—the 
coaches’ qualifications or attributes that make the coaching successful. The most common 
positive attributes of coaching that the staff reported were related to the coaches’ interpersonal 
skills and being in the role of emotional supporter. For example, one staff member said that she 
appreciated having “the extra person there saying [she is] doing a good job.” Another teacher 
reported that the coach’s support helped her feel better about her work, making her more positive 
and motivated to improve her teaching. The second most frequent component addressed the 
coach’s availability to help the staff, and the responsiveness in providing answers or resources. 
One teacher noted that “someone is there to ask you if you need any help because sometimes 
that’s what you need. You need to know that somebody’s there that you can go and ask if you’re 
doing something okay or if you need more help.” The respondent with the lowest perception of 
coaching success felt that coaching as a professional development strategy was not right for her 
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at the time, but she did not have a choice about working with the coach. Although this 
respondent noted that she had heard that other teachers appreciated their coaches and she knew 
that a lot could be learned from the coach, she personally did not want the experience. 


Exhibit 51. Types of Skills for Effective Coaching - Staff Responses 


Component n Percentage 


Provide emotional support from coach 23 29.1% 
Be availability/responsive to coach 16 20.3% 
Provide constructive feedback from coach 15 19.0% 
Be active in the classroom 14 17.7% 
Provide instructional guidance and assistance 11 13.9% 
Provide new ideas for the classroom 13 16.5% 
De management 4 5 1% 


Note. The source is the staff telephone interview. Seventy-nine staff members reported data related to this 
theme. However, there is duplicate counting because 24 individuals reported more than one component. 


When staff talked about how their work changed as a result of coaching, a range of areas were 
discussed, as the thematic categories in Exhibit 52 illustrate. The most common change that the 
staff identified was a change in instructional practices. As with the coaches, this change was 
further distinguished into two subcategories. Some of the staff noted changes in their tools of 
instructional practices, including the use of new activities such as letter boards and more hands- 
on materials (n = 15). Other staff members tended to report a change in their approach to 
instructional practices (n = 14). This refers to a change in the staff level of intentional planning 
and awareness of instruction in the classroom. One staff person shared that “coaching made me 
more intentional and dynamic. I have great ideas, but you have to step back and think about how 
to include each child.” This staff person focused on scaffolding to make sure that children got the 
most out of reading time and noted that she started “planning deeper” for the potential “what ifs” 
of a situation. 


Exhibit 52. Staff Perceptions About Their Changes in Work as a Result of Coaching 


Goals n Percentage 
Change in instructional practices 35 44% 
Change in work with children, such as more intentional conversations 31 39% 
Change in behavior management abilities 26 33% 
Change in work with families, such as increased communication 21 27% 
Change in work with staff, such as closer relationships 20 25% 
Change in personal outlook of work 14 18% 
Increased or more thorough use of assessments 13 17% 
No change in work 9 11% 
Note. The source is the staff telephone interview. Seventy-nine staff members reported data on this theme. However, there is 
duplicate counting because 49 individuals reported more than one answer. Six staff members did not specify in detail how their 
instructional practices changed. 
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Perceived Challenges 


Just as it is important to understand perceived areas of quality, success, and effectiveness of 
coaching, it is also important to understand its challenges. In the sections below we report on 
challenges expressed from the Grantee Survey, challenges from coaches from their survey and 
interview reports, and challenges from the perceptions of staff captured in their interviews. 


Grantee Reported Challenges. As shown in Exhibit 53, most of the grantees reported at least some 
challenges related to staff and staffing in the survey, but relatively few reported these issues as 
always being a challenge. Among the staffing issues reported in the survey, most of the grantees 
reported having at least some issues with staff openness to improvement (88.2 percent reported that 
this at least sometimes challenging), demands on staff time interfering with scheduling (87.2 
percent reported that this is at least sometimes challenging), and staff level of engagement or 
interest (85.7 percent of the grantees reported that this was at least sometimes challenging). The 
grantees were least likely to report challenges with coach turnover (33.9 percent of the grantees 
reported that this was at least sometimes challenging) or the qualifications, the skills, and the 
abilities of the coaches (33.1 percent of the grantees reported that this was at least sometimes 
challenging). There was a fair amount of variability in the grantee reporting of challenges for the 
basic skill levels of the staff being coached, staff turnover, and the number of staff per coach. 


Exhibit 53. Grantee Report of Staff or Staffing Challenges to the Success of the ELMC Initiative 


Never 


Sometimes 


Often 


ATEN 


Challenging 


Challenging 


EU Carellare 


EU tellare 


Staff level of openness to self-improvement 11.8% 69.7% 16.8% 1.7% 
Demands on staff time interfering with scheduling 12.8% 40.2% 35.9% 11.1% 
Staff level of engagement or interest 14.3% 74.8% 9.2% 1.7% 
Basic skill level of staff being coached 21.4% 66.7% 11.1% 0.9% 
Program staff turnover 37.1% 42.2% 17.2% 3.4% 
Number of staff per coach 38.1% 35.6% 20.3% 5.9% 
Coach turnover 66.1% 24.6% 5.1% 4.2% 
Qualifications, skills, and abilities of the coaches 66.9% 30.5% 0.8% 1.7% 


Note. The source is the grantee census survey. The sample size ranges from 116 to 119 grantees. 


As shown in Exhibit 54 one of the biggest challenges for the grantees to the success of the ELMC 
initiative was the availability of substitutes for the staff, with 80.2 percent reporting that this was at 
least sometimes challenging. Almost two-thirds of the grantees reported issues—sometimes, often, 
or always challenging—with travel between centers (63.2 percent). There was a fair amount of 
variability in the grantee report of technological issues, the ability to provide CEUs for 
participation in coaching, and the adequacy of financial resources. Otherwise, relatively few of the 
grantees reported major challenges (often or always challenging) with other issues. 


Exhibit 54. Grantee Report of Other Challenges to the Success of the ELMC Initiative 


Never 


Sometimes 


Often 


PEWS 
Challenging 


Availability of substitutes for staff 


Challenging 
19.8% 


Challenging 
38.8% 


Challenging 
26.7% 


14.7% 


Travel issues (distance between centers) 


36.8% 


44.4% 


10.3% 


8.5% 
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Never Sometimes Often PENS 


Challenging Challenging Challenging Challenging 


Technological barriers (e.g., Internet access) 49.2% 34.7% 13.6% 2.5% 
Openness of community to trusting coach 49.6% 45.3% 4.3% 0.9% 
Availability of space for coaching meetings 56.0% 34.5% 7.8% 1.7% 
Ability to provide CEUs for participation 56.5% 19.4% 74% 16.7% 
Center-level administrative support and involvement 56.8% 35.6% 7.6% 0.0% 
Availability of supplies and resources 63.2% 29.9% 6.0% 0.9% 
Adequacy of financial support and resources 63.2% 26.5% 6.0% 4.3% 
a community buy-in to quality improvement in 64.7% 32.8% 2.6% 0.0% 
Adequate supervision for coaches 75.2% 22.1% 1.8% 0.9% 
Grantee-level administrative support and involvement 81.4% 17.8% 0.8% 0.0% 


Note. The source is the grantee census survey. The sample size ranges from 108 to 118 grantees. 


Coach Reported Challenges. As shown in Exhibit 55, we asked coaches to report on different 
types of contextual factors that could be challenging to their success as coaches. Many of the 
coaches reported that program staff turnover was a challenge at least sometimes (64.6 percent). 
Both grantees and coaches reported staff-related issues such as staff being available and present 
for coaching as one of the biggest challenges to the success of the ELMC initiative. About one- 
third of coaches noted that the level of support from the HS director could also be challenging, at 
least sometimes. Less than 15 percent of the coaches faced challenges with support from other 
coaches or families’ resistance to coaching presence in the home. 


Exhibit 55. Coach Report of Contextual Factors That Were Challenging to Their Success 


Never Sometimes Often PEWS 
Challenging Challenging Challenging Challenging 
Program staff turnover 35.4% 46.0% 13.2% 5.3% 
Level of support from HS or EHS director 66.6% 25.2% 6.1% 2.1% 
Level of support from other coaches 85.4% 12.5% 1.9% 0.3% 
Families’ comfort with a coach in their homes 85.6% 12.0% 2.1% 0.3% 


Note. The source is the coach census survey. The sample size ranges from 326 to 378 coaches. The low response rate of 326 coaches was for the question 
about families’ comfort with a coach in their homes. 


We explored more closely the issue of the staff being coached as a challenge to the success of 
providing coaching, in both the survey and the interviews with coaches. In the survey, coaches 
were asked to consider how challenging various staff characteristics. In addition to these staff 
characteristics, we asked about the relationship quality between the coach and the staff being 
coached and the match (e.g., personality, age, and experience) between the coach and the staff. 
The response options included never, sometimes, often, and always challenging (Exhibit 56). 
Most of the coaches reported that staff level of openness to self-improvement was at least 
sometimes challenging (only 12.9 percent reported that this was never challenging), as well as 
the level of effort (only 14.9 percent reported that this was never challenging), and the level of 
staff engagement or interest (only 15.7 percent reported that this was never challenging). Staff 
skills were also a challenge for many of the coaches, including the ability of the staff to use 
feedback (17.7 percent reported that this was never challenging), the ability of the staff to share 
mistakes (23.7 percent reported that this was never challenging), and the basic skill levels of the 
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staff being coached (27.6 percent reported that this was never challenging). 


The majority of the coaches felt that the level of openness, the level of effort, the ability to 
engage in self-reflection, the ability of the staff to use feedback provided by the coach, and the 
ability of the staff to share mistakes were sometimes or often challenging to the coaches’ 
success. However, the coaches were less concerned with community buy-in and trust and the 
relationship quality with staff. They were also least likely to be concerned with the match 
between themselves and the staff, with only 0.5 percent indicating that this was always 
challenging. 


Exhibit 56. Coach Reports of Staff Characteristics That Were Challenging to Their Success 


Never Sometimes Often Always 
Challenging Challenging Challenging Challenging 
Level of openness to self-improvement 12.9% 55.0% 25.0% 7.1% 
Level of effort 14.9% 60.6% 18.6% 5.9% 
Level of ability to engage in self-reflection 15.6% 55.2% 25.2% 4.0% 
Level of engagement or interest 15.7% 59.8% 19.4% 5.1% 
Ability of the staff to use feedback 17.7% 63.2% 16.9% 2.1% 
Ability of the staff to share mistakes 23.7% 53.2% 21.3% 1.9% 
Basic skill level of staff members being coached 27.6% 51.7% 18.6% 2.1% 
Community buy-in to quality improvement in general 33.2% 46.6% 16.4% 3.8% 
Openness of community to trusting coach 34.6% 48.1% 14.9% 2.4% 
Relationship quality with staff coached 45.5% 49.5% 4.0% 1.1% 
Match (e.g., personality and experience) with staff 52.6% 43.4% 3.4% 0.5% 


Note. The source is the coach census survey. The sample size ranges from 375 to 380 coaches. 


As shown in Exhibit 57, the demands on staff time interfering with scheduling were a significant 
challenge for coaches: 21.1 percent reported that this was always challenging (but only 9.2 
percent reported that this was never challenging). Many of the coaches also reported challenges 
with variation in staff needs (71.8 percent reported that this was at least sometimes challenging) 
and methods for identifying staff coaching needs (62.7 percent reported that this was at least 
sometimes challenging). Job demands from work other than coaching were always a challenge 
for 12.3 percent of the coaches and often a challenge for 14.7 percent of the coaches. However, 
other organizational features tended to be an issue for a smaller percentage of the coaches. 


Exhibit 57. Coach Reports of Organizational Systems That Were Challenging to Their Success 


Never Sometimes Often WIENS 


Challenging Challenging Challenging Challenging 


Demands on staff time interfering with scheduling 9.2% 39.6% 30.1% 21.1% 
Variation in staff needs 28.2% 55.9% 13.2% 2.6% 
Methods for identifying staff coaching needs 37.3% 55.8% 6.1% 0.8% 
Job demands from work other than coaching 41.1% 32.0% 14.7% 12.3% 
Number of staff per coach (i.e., caseload size) 46.2% 35.4% 11.6% 6.9% 
Consistency of messaging across coaches 52.1% 38.5% 8.6% 0.8% 
Lack of language match with staff, children, or families 73.1% 23.0% 2.9% 11% 


Note. The source is the coach census survey. The sample size ranges from 374 to 379 coaches. 
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As shown in Exhibit 58, many of the coaches reported that the availability of substitutes for staff 
was a Challenge: 16.3 percent reported that this was always challenging, 25.0 percent reported it 
was often challenging, 36.3 percent reported it was sometimes challenging, and 22.4 percent 
reported that it was never challenging. Other logistic factors tended not to be a major issue for a 
majority of the coaches. 


Exhibit 58. Coach Reports of Logistical Factors That Were Challenging to Their Success 


Never Sometimes Often WIENS 
Challenging Challenging Challenging Challenging 

Availability of substitutes for staff 22.4% 36.3% 25.0% 16.3% 
Availability of supplies or resources 47.6% 41.3% 8.5% 2.6% 
Technological barriers (e.g., Internet access) 50.3% 35.7% 10.1% 4.0% 
Availability of space for coaching meetings 51.3% 34.7% 10.5% 3.4% 
Travel issues (distance between centers) 53.6% 33.0% 8.4% 5.0% 
Availability of CEUs for staff 68.1% 22.3% 6.7% 2.9% 
Languages of staff, children, and families 68.4% 28.2% 3.4% 0.0% 


Note. The source is the coach census survey. The sample size ranges from 373 to 380 coaches. 


A structural aspect of coaching that emerged as a challenge during the telephone interviews was 
reported by coaches who worked at more than one site. Most of the coaches we interviewed (n = 
27; 79 percent) found that providing coaching in multiple locations was challenging. Two types 
of challenges emerged from our interviews: (1) the coaches felt that they did not get to see the 
staff often enough, and (2) the coaches felt that they did not have enough time to work with the 
staff. For example, one coach worked for a grantee that encouraged the coaches to see teachers 
twice per month. However, the coach said this was difficult because of the large distance 
between the rural centers, which were 60 miles or more apart, with some sites being two hours 
away from where she was primarily located. Although there were some who found that working 
in multiple locations was a benefit (n = 17; 50 percent), 10 coaches in the interviews said that 
working in multiple locations was both a challenge and a benefit. 


Exhibit 59 lists the major themes from the coaches’ interviews when challenges were discussed. 
Fifty-three percent of the coaches interviewed discussed more than one challenge in their 
interview. 


Scarcity of Resources. The top challenge described was not having enough resources, either the 
time to do all the work related to coaching or the financial resources that they felt were needed 
(62 percent). As an example, a coach explained that she had two roles with the grantee: one as a 
coach and the other as a family support worker. Being a family support worker was her first and 
primary job; she found the demands of both jobs difficult to manage and she struggled to get in 
the hours of coaching that were necessary. Another illustrative challenge was regarding 
coordination between the coaches and others, such as planning logistics, coordinating efforts, or 
communications (45 percent). For example, one coach stated the biggest challenges to ELMC for 
this coach were the structural issues that were part of this grantee. Teachers were allowed a lot of 
time off, and if one or more teachers had to leave for a funeral or any another reason, the 
classroom or center could be shut down for the day. This troubled the coach because she felt that 
they couldn’t make progress as quickly as they should. In another example, one coach explained 
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that the staff members were getting information from different sources that were not aligned (11 
percent of coaches also talked about the lack of clarity in their roles as a challenge). For 
example, a coach would say one thing but then the director or others would tell the staff 
something else, which undermined staff buy-in. 


Administrative Issues. Twenty three percent of the coaches related to concerns with director 
involvement and support. For example, coaches described issues when the director was not 
supporting classroom goals or when a coach was required to have a high level of involvement 
with every staff member, regardless of need. One coach explained that, as the only coach for 
more than 30 staff members, she did not have the time for a high level of involvement with 
everyone and perceived it would have been easier if the administrator had identified which staff 
needed coaching the most. Another coach stated that in addition to the lack of support, there 
were also technology and communication challenges, regarding scheduling and administrative 
backing of the coaching effort. 


A few of the coaches said they had their own issues that were challenging (6 percent). For 
example, one coach explained she had worked in only one center before becoming a coach, so 
she was surprised by the range in the level of education and skills from staff in different centers 
and different classrooms; she explained that dealing with variation that she didn’t know existed 
was a Challenge for her personally in her work as a coach. 


Exhibit 59. Coach Challenges From the Interview Data 


Challenges n Percentage 

Limited resources: not enough time or money 33 62% 
Coordination: planning, logistics, and 

communications 24 45% 
Staff issues 17 32% 
Administration issues 12 23% 
Coach-staff relationship 9 17% 
Equipment, materials, and space 9 17% 
Lack of clarity about roles and responsibilities 6 11% 
Own issues 3 6% 


Note. The source is the coach telephone interview. Fifty-three coaches reported data related to this 
theme; 28 coaches gave more than answer. 


Staff Reported Challenges. In the telephone interviews, we asked the staff about any challenges 
they perceived in the coaching they received. Overall, most of the staff were positive and did not 
explicitly describe or provide examples about any challenges. However, a small number of staff 
members discussed what they found to be challenges or, more specifically, ineffective 
components of the coaching they received. Six staff members specifically mentioned that the 
content or the feedback they received was inadequate (n = 6) or did not lead to constructive 
action (n = 5). For example, one teacher explained her frustration with receiving inadequate 
coaching by saying “a mentor should be a really strong person. This coach seemed wishy-washy 
about things and always wanted to be positive.” This staff person expressed that she wanted to 
have a coach who was able to provide both praise and constructive criticism. Another teacher felt 
that the goals of coaching were not explicit enough to be helpful. Other challenge themes raised 
by less than a handful of the staff were inadequate frequency with their coaching sessions (n = 
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3), the coach lacking knowledge (n = 2), or coaching that was a burden to the staff (e.g., time 
demands) (n = 2). 


Factors Associated with Successes and Challenges 


Perceptions of the success and challenges of coaching may be related to a range of dimensions, 
including contextual, structural, and process. We ran zero-order correlations to identify whether 
these coaching dimensions were linked with a number of constructs, including with: 


e Coach ratings of successes and challenges; 


e The independent variables (i.e., the number of national professional development sources 
used and the coach’s qualifications, such as education level and years of experience 
teaching or training adults, in ECE, or as an ECE coach); and, 


e The dependent variables (i.e., the coach ratings of their success at improving the quality 
of staff practices, their overall success as coaches, and the total number of challenges 
reported by the coaches). 


The correlations were small (all under .15), and no correlation was significantly different from 
zero. The analyses suggest that these inputs had little influence on the coach perceptions of 
success, overall challenges or use of professional development resources. °° 


Summary of Perceived Outcomes of Coaching, Successes, and Challenges 


Most of the coaches reported success in increasing staff openness to learning and improving the 
quality of practices. The staff reported changes in both instructional and behavioral management 
practices. Both the coaches and the staff reported that their relationships were supportive and 
open. 


Overall, the grantee administrators were very positive about the coaches. The grantees were most 
enthusiastic about the coach success in training and supporting staff and in increasing staff 
openness to learning. The coaches were fairly positive about their success in specific aspects of 
their work with the staff, and were most likely to describe that their work improved the ways that 
the staff interacted with children. Virtually all staff provided positive feedback on their 
experience with coaching. The majority of staff noted that coaching was very effective, reported 
overwhelmingly positive perceptions, and were willing to continue coaching. 


Common challenges of the coaching from grantees and coaches tended to relate staff issues, such 
as turnover rates or staff time demands or lack of substitutes to cover classes so they could 
engage in the coaching process. One of the biggest challenges to the success of the ELMC 
initiative reported in the grantee survey was the availability of substitutes for the staff. 


°° Additional details about correlational analyses are in Volume 2: Appendixes, Appendix E. 
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Sustainability of Coaching Programs 


The majority of the grantee survey respondents (71.7 percent) indicated that it was very likely or 
moderately likely that the grantee would continue with coaching after the ELMC grant funding 
ended. Of the remaining grantee respondents, 17.7 percent reported that it was somewhat likely 
that they would continue coaching, and 9.9 percent reported that it was not at all likely. 
Meanwhile, 45.4 percent of the coach respondents indicated that they expected to continue 
working for the grantee as coaches after the ELMC grant ended. 


Most of the grantees reported that they had taken steps to continue with coaching after the 
ELMC grant ended, but the specific steps varied. Among the 102 grantees expecting some 
likelihood of continuing the ELMC grant, only 5.9 percent indicated that no steps had been taken 
yet to sustain the coaching programs. 


Although most of the grantees reported that they had taken steps to continue coaching in the 
survey, the subgroup analyses found that small and rural grantees were less likely than large and 
urban grantees to have made any efforts toward sustainability. Although the results of the 
subgroup analyses with small and rural grantees are similar, about one-half of rural grantees are 
large, so these subgroup differences are not the result of the subgroups representing the same 
group of grantees. 
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Overview of Key Findings 


We present the study findings according to key dimensions of coaching listed in Exhibit 4 (page 
17). These dimensions include: (1) the context of coaching, (2) basic dimensions of coaching, (3) 
structural dimensions of coaching, (4) procedural dimensions of coaching, (5) outputs of 
coaching, (6) perceived outcomes of coaching, successes and challenges, and (7) sustainability. 


Overall, the ELMC descriptive study found that a diverse group of HS grantees participated in 
the initiative, encompassing both large and small programs (ranging from serving fewer than 400 
children to serving more than 5,000) in urban and rural settings (ranging from sparsely populated 
rural areas to urban areas with more than one million residents). Forty-five percent of grantees 
offered coaching to their staff prior to the receipt of the ELMC grant. 


Programs reported on their overarching goals for the ELMC grants and the qualifications for 
effective coaches. Grantee goals commonly reported for the ELMC grant included improving 
classroom quality and staff practices and addressing practices important for the Classroom 
Assessment Scoring System (CLASS). On average, grantees hired their first mentor coach 2.4 
months after the start of the grant, and 72 percent of grantees had begun hiring within 4 months 
from the grant start date. The grantees ranked interpersonal skills as a key qualification in their 
hiring decisions, along with experience in training, teaching, mentoring, or coaching adults. 
Consequently, the coaches as a whole were highly educated and had many years of ECE 
experience. Coaches also rated interpersonal skills as the most important coach qualifications 
followed by a background in ECE, a degree in ECE, and the ability to provide constructive 
feedback. 


Most of the coaches worked part-time, although about one-third worked at least 40 hours a week. 
About half of the coaches were internal hires and about half were hired as temporary staff or as 
external consultants. Approximately 20 percent of the coaches held another job position with the 
grantee in addition to their coaching position, and about 44 percent of the coaches reported 
spending at least some time each week doing work for the grantee that is not part of their 
coaching role. 


Coaches worked with a remarkably broad array of HS program staff. Classroom teachers and 
assistant teachers most commonly received coaching, though 19 percent of staff were home 
visitors and 18 percent were supervisors or administrators. Most of the coaches who worked with 
teaching staff worked with both members of the teaching team (lead and assistant teacher). The 
coaches commonly worked in multiple centers; however, about one-fourth worked in only one 
center, while almost 40 percent worked in two to four centers and about one-third worked in five 
or more centers. About one-half of the coaches worked with 10 or fewer staff, whereas about 
one-fourth of the coaches worked with more than 20 staff. 


When asked about supervisory roles, about three-fourths of the coaches did not have supervisory 
responsibility for the staff that they coached. However, almost all of the coaches reported to 
someone at the grantee level with regard to the overall progress their staff was making. Staff did 
not perceive that coaches serving as supervisors or reporting to their supervisors was a problem. 
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Some reported that it helped keep everybody on the same page. 


A key component of the coaching process is the focus of the individual coaching interactions. 
Coaches made decisions about coaching topical targets in varied ways. Staff generally self- 
identified their coaching needs, and the coaches also identified staff needs by observing staff at 
work and using both formal and informal assessments. Asked to rate topics for coaching, the 
coaches reported their top three areas as: (1) the general skills and strategies of the staff, (2) the 
program and classroom operations, and (3) the use of assessment and technology. The staff 
reported their coaching goals as: (1) improving the physical environment of the classroom, (2) 
improving teacher quality, (3) improving teaching of a school-readiness topic, (4) improving 
CLASS scores, and (5) providing behavior management techniques. 


The coaches generally perceived their roles as being collaborative partners with the staff they 
supported, while many also noted that they provided emotional support and instruction. The staff 
perceived the coaches as assistants, advocates, and sources of emotional support. The coaches 
were most likely to report using on-site observation, verbal feedback, and reflective strategies 
with each staff person at least three times in a typical month. Very few reported using video 
strategies. 


Grantees, coaches, and staff were asked about their perceptions of the coaching. Overall, the 
grantee administrators were very positive about the success of the coaching effort. Most of the 
coaches reported success with increasing staff openness to learning and improving the quality of 
practices. The staff reported changes in both instructional and behavioral management practices. 
Both the coaches and the staff reported that their relationships were supportive and open. 


Virtually all staff receiving coaching provided positive feedback on their experience. The staff 
noted that coaching was very effective and reported overwhelmingly positive perceptions. Most 
of the staff were willing to continue coaching. Staff identified emotional support, availability or 
responsiveness, and constructive feedback as the most effective components of the coaching. The 
majority of the grantee administrators indicated it was very likely that they would continue to 
provide coaching after the ELMC grant ended. 


Despite general satisfaction, there were also implementation challenges for many grantees. The 
primary challenges to the success of the coaching effort included limited time and limited 
financial resources. Coaches also reported that variation in staff needs and methods for 
identifying those needs were challenging, along with staff level of openness to self-improvement 
and level of engagement in the coaching process was at least sometimes challenging. Finally, 
both grantees and coaches reported that the management of coaching logistics, such as 
scheduling and finding substitutes, was a challenge. 


This study has several limitations. It is not representative of all HS programs because the ELMC 
grants were awarded competitively to a small pool of HS programs. In addition, data collection 
started in the final months of the official grant period, so the study was limited in both the type of 
data it could collect and the questions it could address. Given the design limitations, any findings 
in this report should not be interpreted as causal links, and caution should be taken when 
considering the applicability of the findings to Head Start more generally. 


American Institutes for Research The Descriptive Study of the Early Learning Mentor Coach Grant Initiative—75 


A Program Logic Model for Coaching 


One task of this descriptive study was to develop a conceptual model for coaching in HS 
programs to serve as a framework for programs seeking to design and implement their own 
coaching initiatives. At the outset of the study, the research team used its expertise in early 
childhood, coaching, and HS programs, along with a review of the early childhood and K-12 
literatures, to identify the dimensions of coaching that might be relevant for describing the 
coaching approaches adopted by the ELMC grantees. We then developed a program logic model 
to identify aspects of coaching approaches and factors affecting their implementation that we 
sought to capture in the descriptive study. 


We learned a tremendous amount about how coaching was structured and implemented under the 
ELMC initiative. However, there were certain dimensions and aspects of coaching that were not 
feasible to address within the constraints of the present study. For example, given the timing of 
the project, direct observation or activity logs of the coaching process were not possible. This 
limited our ability to understand what actually transpired during coaching sessions—for 
example, specific content covered, time spent on each topic, the coaching strategies used, and the 
time spent on the coaching process (versus more logistical tasks). It was also not possible for this 
study to obtain objective assessments of the outcomes directly targeted by coaching, including 
staff knowledge, skills, strategies for working with children and families, and the quality of the 
classroom environment. It was also beyond the scope of this study to examine the child and 
family outcomes expected to improve as a result of staff coaching. 


With input and advice from federal staff and an expert consultant group and by drawing on 
recent research, we refined the program logic model originally developed to guide the ELMC 
descriptive study so that it would have broader application to all HS programs considering a 
coaching program and for researchers tasked with evaluating these programs. The resulting 
program logic model is shown in Exhibit 60. (For a more in-depth description of this program 
logic model and its components, see Putting the Pieces Together: A Program Logic Model for 
Coaching in Head Start. From the Descriptive Study of the Head Start Early Learning Mentor 
Coach Initiative (McGroder et al., 2013). 


According to this conceptual model, the ultimate goal of any program improvement efforts— 
including coaching—is positive, significant, and sustained outcomes for HS children and their 
families. Achieving this ultimate goal depends on the degree to which the structure and process 
of coaching improves targeted staff outcomes. The success of coaching is, therefore, dependent 
on a program administrator’s underlying assumptions regarding why coaching is needed (e.g., 
to improve instructional practices), which staff need or could benefit from coaching (e.g., novice 
versus veteran staff), the qualifications sought in coaches (e.g., interpersonal skills and 
experience coaching adults), and how long it takes to change staff practices and behavior (e.g., 
months or years). These assumptions drive all subsequent decisions—from the “basics” of 
selecting who will receive coaching and whom to hire as coaches, to how best to structure the 
coaching initiative to foster effective coaching processes. 
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Exhibit 60. Program Logic Model for Coaching in Head Start 


ULTIMATE GOAL: Positive, significant, sustained outcomes for children and their families 
eee INPUTS, COACHING CHILD AND 
(> DECISIONS)| | RESOURCES APPROACH OUTPUTS TARGETED OUTCOMES FAMILY OUTCOMES 
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PROGRAM CONTEXT: Characteristics of staff, teaching environment, and organization (staff turnover, culture/climate, values, priorities, 
support for and history of quality improvements, emphasis on performance monitoring) 


: i : : i : 


COMMUNITY CONTEXT: Characteristics of child, family, and community 


As shown in Exhibit 60, assumptions also drive decisions about resources to dedicate to the 
coaching initiative and how to structure the coaching approach. The primary resource for any 
coaching initiative is the coach and the experience, education, and interpersonal skills that they 
bring to the task. Resources also include the space, technology, and a variety of internal and 
external training supports available to the coaches. 


Grantee decisions regarding how many and which staff would receive coaching and the duration 
of the coaching initiative influenced the structure of the coaching approach, including the 
desired coach-staff ratio, the geographic distribution of the staff receiving coaching, and the 
resulting number of coaches required. Structural dimensions also included where the coaching 
took place (in the center, in the home, or off-site), other staffing decisions (whether coaches were 
employees or consultants, and whether employed part- or full-time), and how coaches were 
supervised. 


According to the practice-based model proposed by the National Center for Quality Teaching 


and Learning, the process of coaching is cyclical and consists of three interrelated components: 
(1) identifying needs, establishing goals, and outlining action steps aligned with these needs; (2) 
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engaging in focused observation of current practices and behavior; and (3) fostering reflection 
and providing feedback on practices and behavior (Snyder et al., 2012). Each component 
involves a variety of strategies and materials that coaches can use with varying degrees of 
frequency, depending on the individual needs and the improvement goals of the staff. Coaches 
may adopt one or more roles—from technical expert to collaborative partner to problem solver 
(Lloyd & Modlin, 2012)—based on what the staff need and how they might best respond to 
feedback. Depending on how the initiative is structured and which resources are available, the 
coach may also play a supervisory role, either serving as the staff’s supervisor or reporting on 
staff performance and progress to their supervisors. 


Outputs of coaching refer to the “dosage” received by staff with respect to time actual spent in 
the process of coaching (e.g., discussing needs, being observed, reflecting on practices, receiving 
feedback and resources), as well as the time spent on various topics. The structure and the 
process of coaching will affect the dosage received. For example, coaches who provide modeling 
and practice opportunities may need more time for their sessions. 


The outputs of coaching can also include the quality of the coach-staff relationship and staff 
engagement in coaching—including the degree to which the staff are open to self-improvement, 
make efforts to improve, and are coachable (i.e., are able to self-reflect, share mistakes, and use 
feedback). Once again, this is cyclical. As shown by the double arrows in Exhibit 60, the 
coaching process can affect the coach—staff relationship and the staff’s engagement in the 
coaching process, but staff engagement and the coach-staff relationship can also have 
implications for the effectiveness of the coaching process. 


Sufficient dosage of coaching should improve the proximal outcomes targeted by coaching— 
namely, staff knowledge, skills, and strategies for working with children and their families; the 
quality of the classroom environment; and staff’s pursuit of additional professional development, 
education, and training. 


If coaching is successful in improving these targeted outcomes, then one should see 
improvements in child and family outcomes aligning with the goals and content of the 
coaching. For example, if coaching focused on strategies for improving children’s literacy and 
language development, then one would expect to see improvements in these child outcomes. 
Alternatively, if coaching focused on how best to empower and engage parents in their children’s 
learning, then one would expect to see improvements in parents’ abilities to support their 
children’s learning and development. 


As with any intervention, the successful implementation and effectiveness of coaching may 
depend on the immediate program context, including organizational support for and history of 
quality improvements. 


Also potentially critical is the broader community context, such as characteristics of the 
children and families served and community resources. These contextual factors may not only 
shape the coaching initiative and the outcomes achieved, but the success and implementation of 
the initiative itself may affect the grantee organization—affecting the sustainability of coaching 
and leadership’s openness to future quality improvement efforts in general. 
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Suggestions for Future Research 


The descriptive study of the ELMC initiative sought to describe the various aspects of coaching 
adopted by the ELMC grantees. The study findings and the conceptual framework raise 
additional questions that suggest fruitful areas for additional research. For example, it would be 
helpful to explore the combinations of strategies used by each coach, to discover if certain 
coaching strategies tend to go together, and whether these different patterns can be considered 
discernible program models. Such information could inform future efforts to customize coaching 
specific to the ECE setting (e.g., center-based classroom, home-based family visits, child-care). 


It would also be helpful to have a better sense of the range of expenses and costs to develop and 
implement a coaching program. This study did not have any information about the costs of the 
program, the sources of money HS grantees may have been using in addition to their ELMC 
grant, resources for sustaining it, or any other cost information. 


Future studies of coaching should also examine what actually transpires during a coaching 
session to understand the experience of coaching so one can learn more about the process of 
coaching and address questions about how coaching works, for whom it works, and under what 
circumstances. In other areas of ECE research focused on relationship-based interventions, such 
a line of inquiry is sometimes referred to as “getting inside the black box” of the intervention 
(Emde, Korfmacher, & Kubicek, 2000; Korfmacher, 2001). For coaching research, there are 
many dimensions and interactions that are under-examined, including the understanding of the 
coach-staff interactions and the relationship-based process that influences changes in behavior. 
Spending more systematic effort in getting information about the extent to which staff 
knowledge, attitudes, and practices are changed and more qualitative detailed information about 
staff perceptions about the coaching process would also be useful. 


In addition, future research could use the evaluation logic model resulting from this project to 
design studies to empirically test the causal links within a coaching approach. The unique effects 
of different coaching components on outcomes have not been not systematically varied to 
determine their independent, additive effects or interactions. Little evidence exists on the specific 
components or dimensions of coaching programs that were related to specific outcomes. For 
example, it would be useful to know what components of coaching dosage are, in fact, related to 
improvements in staff knowledge, skills, and practices. Or what type of coaching format (in- 
person vs. group) produces the largest impact on teacher behavior. This type of research could 
suggest to programs where to focus their efforts when seeking to maximize the positive effects of 
coaching. 


These are merely some examples of additional research areas to be explored as derived from the 
ELMC descriptive study. The grantee administrators, coaches, and staff contributing to this study 
demonstrated coaching as applied in the “real world” early learning field. Their experiences and 
perceptions provided the groundwork for future coaching research, policy, and practice in the 
early childhood settings and the resulting improvement of services to young children. 
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